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INTRODUCTION 


The  USAF  currently  f^esees  the  continuing  need  for  the  KC-135  aircraft  fleet  and, 
ctHisequently,  has  embarked  on  an  avionics  modernization  program.  The  Strategic  Air 
G)mmand  (SAC)  issued  a  Statement  of  Need  (SON,  1987)  addressing  its  need  for 
noodemizing  the  KC- 135  cockpit  avionics.  SAC  stated  the  avionics  systems  currently 
installed  ne^  updating  since  a  large  percentage  of  the  current  avionics  are  late  1950s  and 
early  1960s  technology  (e.g.,  vacuum  tubes,  mechanical  analog  displays).  Consequently, 
die  outdated  technology  has  degraded  the  efficiency,  reliaUlity,  maintainability,  and  safety 
of  the  KC!-135  mission. 

The  Avionics  Modernization  Program  (AMP)  represents  a  shift  in  philosophy  from 
reacting  to  urgent  deficiencies  to  planning  for  anticipated  requirements.  The  long-range  goal 
of  AMP  is  to  develop  a  systematic  dme-phased  avionics  integration  by  means  of 
modification  blocks  (Mod  Blocks)  that  vlill  ensure  all  avionics  upgrades  are  installed  in  a 
manner  that  will  optimize  follow-on  upgrades.  This  integration  plan  emphasizes  the  use  of 
modem  technologies  while  maintaining  commonality  widi  other  Air  Force  weapon  systems. 

Current  technology  allows  the  KC-135  w  be  updated  with  avionics  that  have 
significantly  hi^er  levels  of  reliability  and  maintainability,  thereby  reducing  life-cycle  costs 
ai^  increasing  mission  reliability.  A  fully  integrated  avionics  system  can  alk>  allow  for 
increases  in  mission  management  efficiency  and  automation,  simplified  crew  interfaces,  as 
well  as  enhanced  navigation,  and  reductions  in  overall  crew  worUoad.  Accordingly,  the 
reduction  of  the  current  crew  configuration  (pUot,  copilot,  navigator,  and  boom  opmtor)  to 
that  of  a  crew  with  no  navigator  might  be  feasible  resulting  in  an  additional  savings  in 
manpower  costs. 

The  possibility  of  a  KC-135  crew  reduction  has  also  been  addressed  several  times 
in  the  past.  Gieselhart,  Schiffler,  and  Ivey  (1976)  ctxiducted  a  series  of  flight  tests  with 
dual  Inertial  Navigational  Systems  (INSs)  install^  aboard  a  test  aircraft  They  concluded 
that  with  this  minor  upgrade  plan,  woikload  was  too  excessive  for  the  exclusion  of  the 
navigator.  Schiffier,  Geiselhart,  and  Ivey  (1976)  reviewed  task  analysis  documents  and 
performed  flight  tests  before  similarly  concluding  that  a  four-person  crew  was  necessary. 

However,  more  recent  efforts  have  shown  tiiat  crew  reduction  can  be  accomplished 
effectively.  For  example,  Schiffler,  Geiselhart,  and  Griffin  (1978)  used  flight  tests  to 
demonstrate  the  C-141  aircrew  could  be  reduced  from  its  crew  of  five  to  four  without 
significant  mission  degradation.  Accordingly,  the  C- 141  aircrew  was  reduced  in  size  and 
currently  has  no  navigator  on  board.  In  1981,  Barbato,  Sexton,  Moss,  and  Brandt  used  a 
KC-135  cockpit  design  study  to  determine  avionics  control  and  display  criteria  needed  to 
successfully  accompUsh  a  mission.  Their  study  used  complete  state-of-the-art  systems  to 
conclude  a  KC-135  crew  reduction  was  feasible  given  a  completely  updated  and  redesigned 
cockpit  Madero,  Barbato,  and  Moss  (1981)  us^  the  previous  study  and  subsequent 
system  evaluations  to  develop  a  composite  configuration  that  was  evaluated  in  a  full  mission 
simulation.  They  concluded  the  KC-135  mission  could  successfully  be  accomplished  using 
a  three-person  crew.  Although,  the  stutties  cited  indicate  mixed  results,  the  more  recent 
studies  call  for  a  completely  redesigned  cockpit  as  necessary  for  mission  accomplishment 
given  a  crew  reduction. 
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CSEF  Tasking 

The  Crew  Station  Evaluation  Facility  (CSEF)  operated  by  the  Aeronautical  Systems 
Division  Human  Factors  Branch  (ASD/ENEC^  conducts  real  time  engineeiing  simulation 
evaluations  in  support  of  weapons  system  development.  System  Program  Offices  (SPOs) 
use  the  CSEF  as  an  engineering  tool  for  quantitatively  and  qualitatively  assessing  the  flight 
crew's  performance  as  a  function  of  crew  complement,  crew  station  configuration,  and 
operational  mission. 

As  part  of  the  KC-13S  Avionics  Modernization  Program,  a  Memorandum  of 
Agreement  (MOA)  between  the  Directorate  for  Bomber  and  Tanker  Programs  (ASD/SDB) 
and  the  CSEF  was  signed  in  October  1990.  The  CSEF  was  tasked  with  exploring  the 
feasibility  of  a  reduc^  crew  size  by  developing  an  advanced  cockpit  design  and  avionics 
update,  then  denoonstrating  the  feasibility  of  that  design  in  a  full  mission  simulation 
environment.  The  CSEF  acccnnplished  this  tasking  in  a  three-phase  effort:  (1)  Function 
analysis  phase  (Volume  1),  (2)  Cockpit  design  phase  (Volume  2),  and  (3)  Test  and 
evaluation  phase  (Volume  3).  The  ta^  analysis,  design,  and  evaluation  efforts  were,  by 
intent,  separate  efforts  conducted  to  identify  items  of  interest  such  as  workload  bottlenecks, 
safety-critical  tasks,  etc. 


Function  Analysis  Phase 

The  function  analysis  phase  focused  on  breaking  up  tasks  into  categories  and 
describing  the  who/what/where/why/how  of  each  function/task  (Ward,  Dudley,  Hassoun, 
Hughes,  Rueb,  &  Conroy,  1991;  Vol.  1).  The  crew  positions  were  twoken  up  into  duties, 
large  segments  of  work  performed  by  an  individual  crew  member  (e.g.,  celestial 
navigation),  or  into  tasks,  distinct  work  activities  carried  out  for  a  specific  purpose  (e.g., 
alter  heading  line  entry  in  the  navigator's  log).  Additionally,  high  workload  segments  of  the 
mission  were  identified  to  aid  the  design  team  in  (fetermining  areas  to  focus  on.  This 
information  was  gathered  via  literature  searches,  questionnaires,  interviews,  and 
observations.  Ba^  upon  the  information  gathered,  a  working  group  composed  of  two 
pilots,  two  navigators,  two  human  factors  engineers,  and  one  industrial  psychologist 
recommended  how  the  various  duties  and  taslu  should  be  reallocated  an^or  autcnnated 
(functional  requirements).  The  recommendations  of  this  phase  were  used  during  the  design 
phase  for  determining  which  systems/displays  should  be  used. 

CfltkpiL  Design  Ehase 

This  phase  focused  on  designing  a  two-person  conceptual  cockpit  to  eliminate  the 
navigator  from  the  cockpit  (Bamaba,  Rueb,  Hassoun,  Dudley,  Sc  Ward,  1992;  Vol.  2). 

The  design  effort  used  the  functional  requirements  identified  above  supplemented  by  a 
number  of  other  sources.  User/customer  requirements  were  a  major  consideration,  while 
vendors  of  Cockpit  Display  Units,  flight  computers,  etc.,  were  also  consulted  to  ensure 
current  innovative  technologies  were  considered.  In  addition,  several  of  the  design  team 
members  were  experienced  participants  in  earlier  KC-135  (and  other  aircraft)  advanced 
developmental  projects. 

The  final  design  used  a  Rockwell  Intemational-Collins  Avionics  developed  approach 
as  its  baseline  with  significant  modifications  implonented  by  the  CSEF  team.  These 
modifications  were  based  upon  sound  human  engineering  practices  and  comments  from 
preliminary  "quick  looks,"  where  several  KC-135  crewmembers  were  shown  different 
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design  configurations  and  asked  to  comment  and  make  design  and  location 
recommendations.  These  items  were  provided  to  the  test  and  evaluation  team  (Vol.  3)  for 
their  ccmsideration. 


Test  and  Evaluation  Phase 

During  this  phase,  the  CSEF  personnel  attempted  to  demonstrate  the  feasibility  of 
the  two-person  conceptual  cockpit  developed  during  the  design  phase  (Vol.  2).  The  final^ 
design  configuration  was  demonstrated  in  a  simulated  environment  using  operational 
aircrews.  E^h  aircrew  flew  four  different  simulation  missions  with  varying  levels  of  crew 
workload.  Pilot  performance  data,  as  well  as  subjective  questionnaires  and  oral  responses, 
were  collected.  Given  the  feasibility  of  the  two-person  cockpit  configuration,  this  phase 
would  also  determine  the  functional  requirements  necessary  to  keep  workload  levels 
manageable  for  successful  mission  accomplishment. 

This  volume  describes  the  method  and  results  of  the  test  and  evaluation  phase  of  the 
effort.  Design  recommendations,  lessonc  !eamed,  tradeoffs,  and  other  considerations  are 
also  included  to  provide  guidance  to  SPO  engineers  and  managers  when  developing  human 
factOTS  and  crew  station  requirements  for  KC-135  improverrrent  programs,  especially  those 
involving  crew  reductions.  The  approach  used  to  develop  the  cocl^it  desi^  for  simulation, 
including  the  ground  rules  and  assumptions  f(»'  this  development,  is  also  (fiscussed.  For  a 
detailed  description  of  the  design,  refer  to  Volume  2:  Cocl^it  Design  Report  (Bamaba, 
Rueb,  Hassoun,  Dudley,  &  Ward;  1992). 
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METHOD 

Subieas 


A  total  of  10  KC-135  crews  (pilots  (P),  copilots  (CP),  and  navigators  (N))  and  2 
KC-10  crews  (pilots  and  copilots)  were  used.  They  were  qperational  crews  from  various 
air  bases  (Active,  Guard,  and  Reserve)  throughout  the  United  States.  All  of  the  crew 
membo^  were  qualified  in  their  posititxis. 

KC-13S  Pilots 

The  resulting  personal  data  indicated  the  subject  pool  was  relatively  older  (x=30.4) 
and  more  experienced  (average  total  flight  hours=2249  and  average  tcMal  KC-  13S  flight 
hourssl237)  than  the  current  operational  fcxee  based  upon  the  subjective  judgment  of  the 
subjects.  The  pilots'  average  time  since  last  flight  was  9.5  days. 

KC-10  Pilots 

Similariy,  the  KC-10  pilots  were  relatively  older  (x=31)  and  more  experienced 
(average  total  fli^t  hours=2450  and  average  total  KC-10  fli^t  hoursal220),  than  the 
current  operadoi^  KC-135  crew  force.  ITie  avmge  time  since  last  flight  was  4.5  days. 
Two  of  the  pilots  had  previous  KC-135  experien<%  and  one  had  been  a  KC-135  navigat(»^ 
prior  to  becoming  a  KC-10  copilot 


Apparatus 

Simulator 

The  KC-135  simulator,  shown  in  Figures  1  and  2,  included  such  major 
components  as  the  control  loa^g  assemblies,  seats,  yokes,  and  visual  windows.  The 
simulator  was  quipped  with  two  wide  angle  collimating  windows  that  provided  a 
panoramic  outside  scene  capable  of  supporting  the  CS^  Ni^t  Visual  System  (NVS).  A 
Digital  Equipment  Corporation  (DEC)  PDP  1 1/35  coQq)uter  used  tme  of  a  numl^  of 
dambases  to  generate  sets  of  lights,  simulating  various  night  visual  scenes,  for  the  NVS. 
This  provided  the  sutgects  with  a  visual  capability  used  during  takeoff  and  during  ^proach 
and  landing.  The  KC-135  simulator  cockpit  was  a  newly  designed  conceptual  cocl^it. 

The  actual  instrumentation  used  and  its  description  can  be  foui^  in  Volume  n  (Bamaba, 
Rueb,  Hassoun,  Dudley,  &  Ward;  1992),  which  describes  the  entire  cockpit  for  both  the 
pilot  and  copilot  positions.  The  software  package  contained  all  flight,  en^e,  atmosphere, 
wei^ts  and  balances  modules;  a  dictionary  of  all  KC-135  data  variables;  and  several  other 
specific  commons  and  data  pools  for  the  KC-135A  model  aircraft  In  addition,  a  computer 
Ingram  read  a  Defense  Mapping  Agency  (DMA)  terrain  database  into  memory  of  a  C^uld 
Sel  87  computer.  The  elevation  of  the  terrain  was  then  cranputed  based  on  an  extrapolation 
of  the  simulatCH'  position  in  relationship  to  the  database.  The  subtraction  of  the  terrain 
elevation  fiom  the  aircraft  barometric  altitude  (computer  based)  provided  the  above  ground 
information  fed  back  to  the  aircraft  radar  altimeter  indicator. 
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Computer  Complex. 

The  simulator  was  connected  to  a  series  of  large  and  small  computo:  systems.  This 
conq)uter  con^lex  included  five  Gould  series  32/7780,  one  Gould  concq)t  3^780,  two 
PDP  1 1/34,  th^  PDP  1 1/35,  and  several  Silicon  Graphics  his  Work  Stations.  The 
Silicon  Grsqrfiics  Work  Statims  were  responsible  for  the  EADI/EHSI  visual  presentations. 

Experimenter's  Console 

The  experimenter's  console  (Figure  3),  also  known  as  the  Instructor-Operator 
Station  (lOS),  was  located  approximately  10  feet  away  frwn  the  simulator.  It  included  a 
complete  intercom  system,  with  communication  to  and  from  the  pilots  inside  die  simulator. 
The  console  displays  duplicated  the  simulator  instruments  and  displays,  and  were  used  to 
monitor  the  pik^  as  weU  as  the  simulator  performance .  The  experimenters  were  able  to 
monitor  tfie  CDU,  EHSI,  and  EADI  of  each  pilot  (P,  CP)  through  repeater  displays  located 
at  the  lOS.  Additionally,  the  lOS  provided  the  experimenter,  via  a  data  display  page  (Table 
1),  with  airspeed  (indicated  airsp^-IAS,  true  airspeed-TAS,  and  groundspeed-GS); 
alti^e  (Barometric-MSL  and  radar- AGL);  heading  (true-TH  and  magnetic-MH);  present 
position;  currently  tuned  radios  (UHFl,  UHF2,  HF)  and  navigation  aids  (VORl,  VOR2, 
LCX^l ,  LOC2,  and  TACAN);  date  and  time;  current  waypoint  infcxmation;  current  airprxt 
selected  for  the  visual  system;  and  the  status  of  the  gear,  flaps,  and  boom.  Furthermore, 
tile  consttie  contrrrfs  pomitted  the  experimenter  to  start  the  simulation  via  the  lOS  set-up 
page  (Table  2)  and  to  terminate  the  mission  via  the  lOS  change  modes  page  (Table  3).  This 
page  also  allowed  the  experimenter  to  engage/disengage  the  Ixxxn  and  to  address  the  data 
display  page  and  two  other  lOS  pages,  the  airport  selection  page  and  the  malfunction 
selection  page  (discussed  later). 


Table  1.  An  example  of  the  data  display  page. 


AIRSPEED 
IAS  0 
TAS  0 


ALTITUDE 
BARO  105 
RALT  9 


HEADING 
TRUE  000 
MAG  345 


DATEmME  MISC. 

01/01/01  GEAR  UP 

08:09  FLAPS  0 

BOOM  DISENGAGED 


RADIOS 
UHFl  223.00 
UHF2  227.00 


TACAN 

IDENTIFIER  MHR 
RADIAL  DME  000/  .0 


POSITION 
LAT  N38-33.6.6 
LONG  W121-17.0.0 


DEST.INFO 
WP#  0 
DIST  .0 
TTG  0000 
CRS  000 
OFFST 


DATA  COLLECTION 
MISSION  MATHER  AFB 
AIRPORT  MATHER  AFB 
DATA  INACTIVE 
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SIMULATOR  STATUS 
FLYING 


Press  ATTENTION  to  EXIT  PAGE 


Table  2.  An  example  of  the  IQS  set-uo  page. 


RECORD  AMP  RUNS 


1.  TRIAL# 

2.  SUBJECT  # 

3.  MISSION 


:  4 
:  14 

:  CASTLE  AFB 


A.  INCREMENT  RUN  :  ON 

B.  DATA  COLLECnON :  ON 

C.  FILENAME  :  M003S14V01.D 


PRESS  R'  TO  RUN 


ENTER  COMMAND  ('X'  TO  EXIT): 


Table  3.  An  example  of  the  change  modes  page. 

lOS  PAGE  -  CHANGE  MODES 
ENGAGE  -  ENTER  B 

CHANGE  AIRPORT  -  ENTER  C 

SELECT  MALFUNCTION  -  ENTER  M 

RESUME  MISSION  -  ENTER  R 

TERMINATE  MISSION  -  ENTER  X 


PLEASE  ENTER  YOUR  SELECTION 
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The  primary  objective  of  the  Test  and  Evaluation  phase  was  to  demonstrate  the 
feasibility  of  the  CSEF  conceptual  cockpit  design.  To  accomplish  this,  10  KC-135 
operational  aircrews  from  SAC  air  bases  were  required  to  fly  four  mission  scenarios  over  a 
period  of  5  days  and  2  KC'IO  aircrews  were  required  to  fly  three  mission  scenarios  over  a 
4^y  period.  Since  the  KC-10  crew  had  no  navigate  and  only  flew  the  Mather  and  Castle 
missions,  this  resulted  in  a  repeated  measures  mixed  design  (3  Mission  x  3  Position  x  2 
Plane).  All  missions  were  successfully  completed.  Based  on  initial  direction  by  the  system 
program  office  and  SAC,  several  assumptions  were  made  at  the  very  beginning  of  the 
three-phase  effort  These  were: 

1.  The  missions  were  unclassified  training  missions.  Qassified  Command  and  Control 
Procedures  were  ignored. 

2.  All  mode  1, 2, 3,  and  4  settings  were  assumed  to  be  set  and  correct.  Except  fcH-  the 
mode  3  setting,  no  actual  setting  of  codes  was  performed  by  the  crewmemb^.  The  KIK- 
18  was  simulated. 

3.  Celestial  navigation  was  not  required. 

4.  Global  Positioning  System  (GPS)  and  associated  satellites  were  available  and  used. 

5.  A  dual  Inertial  Navigation  System  (INS)  was  available  and  used. 

6.  (Current  Federal  Aviation  Administration  (FAA)/Air  Forc^trategic  Air  Corrunand 
regulations  and  directives  were  followed. 

7.  Crews  were  familiar  wid^  current  mission  planning  software  used  on  the  Zenith  2^248. 


Training  Materials 

The  following  materials  were  fnivarded  te  the  aircrews  for  study  prior  to  their 
arrival  at  the  Crew  Station  Evaluation  Facility: 

1.  Programrrted  text  describing  the  proposed  design.  It  included  detailed  descripticms, 
standard  operating  procedures,  and  proposed  use  of  the  equipment  to  include  illustraticms 
of  the  Rad^,  Electronic  Flight  Instrument  System  (EFIS),  Electronic  Attitude  Director 
Indicator  (EADI),  Inertial  Navigation  Systems,  Control  Display  Unit  (CDU),  and  Global 
Positioning  System. 

2.  Abbreviated  checklists  were  given  to  each  crew  member. 

3.  All  routes  of  flights  and  associated  mission  paperwOTk  were  also  provided  to  enable 
crew  mission  planning  prior  to  the  crew's  arrival  at  the  CSEF. 


Two  aircrews  (P,  CP,  N)  were  brought  in  for  1  week  at  a  time.  Approximately  1 
week  prior  to  the  arrival  of  the  aircrews,  mission  materials,  training  pamphlets,  and 
checklists  were  forwarded  to  each  crew  for  review  and  study.  On  the  first  day,  each 
aircrew  underwent  training  to  familiarize  them  with  die  desi^  and  qimting  characteristics 
of  the  proposed  system.  Emphasis  was  placed  on  the  crew  understanding  how  to  use  the 


various  subsystems*  in  addition  to  CDU  page  and  radar  scope  interpretation.  The 
following  4  days  were  used  to  fly  each  of  the  4  mission  scenarios  (one  a  day). 

Training 

The  rirst  day  of  the  week  was  designated  as  a  training  day  for  both  crews.  The 
crews  initially  received  a  standardized  bri^g  covering  (1)  the  purpose  of  the  study,  (2) 
the  missions  to  be  flown  during  the  week,  (3)  safety  procures,  (4)  system 
descriptions/operatitms,  and  (5)  their  sch^ule  for  the  week.  Ad^tionally,  all  of  the  pilots 
were  i^uiied  to  perform  a  Subjective  Workload  Assessment  Technique  (SWAT)  card  sort 
(describe  later).  Each  crew  was  then  trained  on  each  of  the  newly  introduced  subsystems. 
They  were  also  provided  flight  simulation  time  so  that  they  would  become  accuston^  to 
the  actual  flight  characteristics/peculiarities  of  this  simulator.  C3EF  personnel  were  present 
at  all  times  to  answer  any  questions. 

Day  2  of  die  week  long  schedule  required  the  crew  to  fly  the  training  mission.  The 
training  mission  provided  the  crew  with  experience  in  the  normal  use  of  the  equipment  and 
in  the  handling  of  various  malfunctions.  It  also  allowed  the  crews  an  opportunity  to  use  the 
equipment  in  a  similar  manner  as  they  would  during  actual  flight,  while  providing  the  crew 
the  qipor^ty  to  interact  widi  the  experimenter,  thereby  gaining  increased  understanding 
of  the  various  pieces  of  equipment 

While  the  pilot  and  copilot  were  receiving  training,  the  navigattnr  was  instructed  on 
the  role  that  he  was  to  play  in  the  study.  The  navigator  aci^  as  an  impartial  observer  inside 
the  cockpit  Since  the  navigator  wori^  with  the  crew  on  a  regular  b^is,  his  presence 
would  not  seem  out  of  the  oxlin^;  whereas,  the  presence  of  an  experimenter  might  prove 
more  intrasive.  He  was  responsible  fcff  recording  any  checklist  omissions  or  deviations 
made  by  the  crew.  The  naviga^  was  also  encourag^  to  assist  die  crew  on  navig^onal 
concerns  at  any  time  during  training.  After  the  training  period,  the  navigator  preyed  no 
further  assistance  to  the  crew. 

Upon  completion  of  training,  die  crew  was  given  mission  materials  for  each  of  die 
3  days.  They  were  then  required  to  conduct  mission  planning  and  prepare  the  associanri 
piqiawoik  for  each  mission.  The  crew  was  told  the  actual  orier  of  the  missions  they 
would  fly  during  diat  week,  but  were  not  informed  as  to  the  difficulty  level  of  each 
mission,  nor  were  they  infonned  as  to  the  actual  sequence  of  events.  The  crew  was 
eiqpected  to  have  completed  all  mission  paperwOTk  imor  to  their  arrival  for  the  flight  At  no 
time  was  the  navigator  to  assist  the  pilot  team  (P,  CP)  in  mission  planning. 

Mission  Simulation 


EUsiita 

Each  crew  airived  at  the  CSEF  each  mission  day  and  checked  Notices  to  Airmen 
(NOTAMS),  the  schedule,  and  the  Flight  Oew  Inftmnation  File  (FCIF).  The  crew  arrived 
with  all  fli^t  equipment  that  they  ordinarily  Inou^t  on  a  regular  flight  with  die  exception 
of  their  helmets  and  flight  lunch^.  The  crew  received  a  weatiier  briefing  (weather  sheets 
were  developed  by  the  Wright-Patterson  AFB  weather  shop  to  enhance  mission  realism), 
cell  brwfing,  and  time  hack  prior  to  being  forwarded  to  the  simulator.  At  their  discretion, 
the  crew  proceeded  to  the  simulator  to  p^orm  the  necessary  checklists  for  scheduled 
takeoff  and  flight 

TaUe  2  presents  an  example  of  the  computer  program  page  that  the  experimenter 
used  to  select  the  mission  flown,  the  subject  crew,  and  ^ta  were  activated. 
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Additionally,  an  lOS  set-up  control  interface  program  developed  to  simplify  user-computer 
interaction  ^owed  the  experimenter  to  monitcH'  i^-time  characteristics  of  the  simulate  as 
it  flew  each  ctxifiguration  (Table  1). 

The  crews  were  required  to  fly  four  different  mission  scenarios  (KC-10  crews  did 
not  fly  the  Minot  Mission).  These  missitxis  were  actual  operational  mission  profiles  flown 
at  Minot  AFB  ND,  Mather  CA,  and  Castle  AFB  CA  and  modified  slightly  for  our 

simulation  efforts.  The  missions  averaged  3  hours  in  duration  and  varied  in  their  degree  of 
difficulty  (easy,  medium,  and  hard).  The  degree  of  difficulty  was  manipulated  through  the 
use  of  weather  factors,  maintenance  problems,  and  mission  r^uirement  changes.  Upon 
entering  the  simulator,  the  crew  immediately  bqgan  their  Prefli^l/Before  Takeoff 
checklists.  The  crews  were  required  to  remain  in  the  coclq)it  tom  pieflight  tluough 
postflight  The  order  and  time  that  each  crew  flew  titeir  3  missions  were  countertelanced 
for  each  mission  (Minot ,  Mather,  Castle)  and  for  time  of  simulation  (morning  vs. 
afternoon). 


Training  Mission.  The  training  mission  was  on  day  2,  the  second  day  of 
training.  The  mission  was  a  three-ship  cell  depamue  and  join-up  tom  Castle  Ai^,  CA. 
The  crew  was  number  two.  At  takeoff  (T.O.)-t-20  minutes,  a  cell  lead  change  was 
performed  and  the  test  crew  became  lead.  Tte  tanker  cell  conducted  an  enroute  rendezvous 
with  a  cell  of  F-ltis  at  the  Air  Refueling  Initiatitxi  Point  (ARIP)  for  AR  route  6B.  The  F- 
16s  required  5,000  (5K)  pounds  more  fuel  than  schedule  Hie  northern  end  of  the  AR 
route  r^uired  the  cell  to  deviate  around  tiiunderstorms.  At  the  end  of  the  refueling  route, 
command  post  notified  the  test  crew  that  they  were  being  diverted  to  McChord  AFB  and 
were  to  await  further  instruction.  Immediately  after  tiie  crew  brolu  up  cell  and  began  its 
alter  heading  to  McChord,  they  experienced  a  generator  failure  and  lost  both  INSs  and  GPS 
equipment  The  crew  was  required  to  navigate  to  McChord  and  land  the  aircraft  (Figure  4). 


Minot  Mission.  This  mission  was  the  "easy"  mission.  The  flight  was  a  single 
ship  mission  departing  tom  Minot  AFB.  The  mission  took  off  9  minutes  late.  The 
receiver  (one  B-52)  arrived  at  the  AR  route  106HW  ARIP  to  an  on-course  rendezvous  3 
minutes  eariy.  The  AR  off-load  was  20K  pounds  mcne  than  scheduled.  The  crew 
experienced  radio  communication  difficulties  with  Center,  therefore,  delaying  End  Air 
Refueling  (EAR)  clearances.  Enroute  to  the  Minot  AFB  Initial  Approach  Fix  (lAF),  the 
crew  experienced  a  hydraulic  failure  (Figure  5). 


Mather  Mission.  This  was  the  "medium"  difficulty  mission.  This  mission  was 
a  two-tiiip  cell  departure  with  an  1  l-minute  late  takeoff  from  Mather  AFB  CA.  At 
T.O.+IO,  Center  directed  the  cell  to  turn  left  direct  to  the  ARCP  for  their  point  parallel 
rendezvous  with  two  B-52  bombers.  At  T.O.+l;15,  INS  drift  was  5  NMPH  when  the 
crew  experienced  GPS  failure.  The  crew  performed  weather  avoidance  due  to 
thunderstorms  on  AR  route  7B  as  required.  Cell  break-up  occurred  at  EAR.  At  EAR,  the 
crew  received  instructions  via  HF  radio  that  they  w^e  diverted  to  Castle  AFB  for 
passenger  ferry.  The  mission  tenninated  with  a  landing  at  Castle  ATO  CA  (Figure  6). 

C:a.<tlc  MUsion.  This  mission  was  labelled  the  "haid''  nussion.  The  crew  was 
number  two  in  a  two-ship  cell.  The  mission  began  at  Castle  AFB  CA  with  an  on-time 
takeoff,  hnmediately  ato  takeoff,  the  crew  experienced  an  autopilot  malfunction.  (>11 
tveak-up  was  scheduled  at  the  Sacramento  TACAN,  prior  to  air  refueling;  however,  the 
lead  aircraft  experienced  hydraulic  failure  immediately  after  cell  departure  and  join-up  was 
performed.  This  left  the  crew  as  a  single  ship.  At  the  Sacramoito  tacan,  tiie  crew  was 
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infonned  that  AR  route  5WE  was  closed  due  to  weather  and  that  they  would  conduct 
random  air  refueling  operations  in  the  Dog's  Tail  Military  Operating  Area  (MOA-W260) 
with  a  single  F-16.  Imm«liately  after  entry  into  Dog's  Tail  MOA,  the  crew  experienced 
complete  loss  of  both  INSs,  in  addition  to  their  GPS  system.  When  the  crew  called  for 
landing  weather,  they  were  given  minimums  that  allowed  for  a  precision  qjproach  to  be 
flown.  However,  the  crew  never  broke  out  of  the  weather  and  was  forced  to  land  at  an 
alternate  airfield  of  their  choice.  Throughout  this  mission  the  crew  was  required  to  deal 
with  an  increased  volume  of  radio  traffic  and  continuous  deviations  due  to  thunderstorms. 
(Figure  7). 

Standardized  mission  scripts  were  provided  to  the  experimenters  to  facilitate  ease  of 
communication  and  provide  a  timeline  for  the  mission  events.  These  scripts  were 
developed  in-house  by  a  former  SAC  operational  navigator  and  reviewed  by  current  SAC 
operations  personnel.  The  scripts  provided  the  proper  sequencing  of  events  in  addition  to 
the  precise  terminology  used  by  air  traffic  controllers,  weather  service  personnel, 
operations  personnel,  maintenance  personnel,  and  other  aircrews.  These  scripts  followed 
the  actual  flow  of  events  and  communications  consistently  across  all  missions  and  all 
crews. 


The  crew  was  required  to  make  all  radio  calls  and  perform  all  actions  that  they 
would  normally  perform  in  actual  flight  This  included  all  stan  engines,  taxi,  takeoff,  and 
cell  formation  calls.  Additionally,  the  crew  was  required  to  clear  any  flight  plan  changes 
with  Center.  Since  the  crew  did  not  follow  scripts  and  was  unaware  of  the  events  until 
actual  occurrence,  experimenters  were  trained  to  listen  intently  to  all  communications  and 
respond  appropriately.  In  an  effort  to  enhance  realism  and  to  reduce  the  monotony  that 
might  ensue  from  hearing  a  single  voice  repeatedly,  it  was  decided  to  use  at  least  two 
experimenters.  Consequently,  two  experimenters,  at  least  one  with  operational  flight 
experience,  acted  as  air  traffic  controllers,  command  post  personnel,  boom  operator,  etc. 

'Die  experimenters  started  the  simulation  via  the  lOS  set-up  page  (Table  2)  when  the 
crew  arrived  at  the  cockpit  Experimenters  continually  monitored  the  actual  position  and 
status  of  the  aircraft  via  the  data  display  page  (Table  1).  The  exp^menter  changed  the 
NVS  airport  visual  database  through  the  use  of  the  airport  selection  page  (Table  4).  This 
allowed  the  console  operator  to  change  the  NVS  without  the  knowl^ge  of  the  pilots.  In 
addition,  it  allowed  the  mission  to  continue  without  interference. 

Table  4.  An  example  of  the  airport  selection  page. 
lOS  -  AIRPORT  SELECTION 


SELECT  TRAINING 
SELECT  MINOT  AFB 
SELECT  MATHER  AFB 
SELECT  CASTLE  AFB 
SELECT  McCHORD  AFB 
SELECT  LOS  ANGELES 
SELECT  DAYTON 
SELECT  SAN  FRANCISCO 
SELECT  BEALE  AFB 


ENTER 0 
ENTER  1 
ENTER  2 
ENTER  3 
ENTER  4 
ENTER  6 
ENTER? 
ENTERS 
ENTER  9 


PLEASE  ENTER  YOUR  SELECTION 


PLEASE  ENTER  AN  X  TO  EXIT  PAGE 


15 


Some  of  the  missions  required  the  console  operator  to  fail  various  systems  onboard  the 
aircraft  simulator.  These  malfunctions  were  implemented  via  the  m^fimction  selection 
page  (Table  5).  This  capability  allowed  the  experunenter  to  induce  workload  at  a 
pr^esignated  time,  thereby,  standardizing  the  mission  profiles  across  crews. 

Upon  completion  of  the  mission,  the  pilots  gathered  their  gear  and  were  lead  to  a  debriefing 
room.  At  this  time,  the  pilots  first  entered  their  SWORD  ratings  (explained  in  the  data 
collection  section)  onto  the  mission  specific  SWORD  data  collection  forms  (Appendix  A). 
They  were  then  administered  a  mission  specific  questionnaire.  After  completion  of  the 
questionnaire  by  all  the  crewmembers,  the  navigator  and  the  experimenter  debriefed  the 
pilots  on  any  errors  made,  suggestions  for  improvements,  and  answered  any  questions 
they  might  have  had.  At  the  completion  of  the  fourth  mission  on  the  last  day,  all  the 
crewmembers  were  required  to  answer  a  system  specific  questionnaire.  The  pilots  were 
then  required  to  perform  a  final  card  SWAT  sort 


Table  5.  An  example  of  the  malfunction  selection  page. 


SELECT  LEFT  HYDRAULIC  OVERHEAT 
SELECT  RIGHT  HYDRAULIC  OVERHEAT 
SELECT  LEFT  SYSTEM  PRESSURE 
SELECT  RIGHT  SYSTEM  PRESSURE 
SELECT  LEFT  AUXILIARY  PRESSURE 
SELECT  RIGHT  AUXILIARY  PRESSURE 
SELECT  LEFT  HYDRAUUC  PUMP  INOP 
SELECT  RIGHT  HYDRAULIC  PUMP  INOP 
SELECT  AUTOPILOT  FAIL 
SELECT  GPS  FAIL 
SELECT  INS  #1  FAIL 
SELECT  INS  #2  FAIL 
SELECT  GENERATOR  #2  FAIL 


ENTER  O 
ENTER  1 
ENTER  2 
ENTERS 
ENTER  4 
ENTERS 
ENTER  6 
ENTER? 
ENTERS 
ENTER 9 
ENTER  A 
ENTER  B 
ENTER  C 


DISENGAGED 

DISENGAGED 

DISENGAGED 

DISENGAGED 

DISENGAGED 

DISENGAGED 

DISENGAGED 

DISENGAGED 

ENGAGED 

DISENGAGED 

DISENGAGED 

DISENGAGED 

DISENGAGED 


PLEASE  ENTER  YOUR  SELECTION 


PLEASE  ENTER  AN  X  TO  EXIT  PAGE 


Data  Collection 

Aircraft  pCTformance  data  collection  was  on  a  1-Hz  cycle.  The  following 
parameters  (with  acronyms)  were  colleaed  continuously  throughout  flight: 

1.  Indicated  Airspeed  (IAS) 

2.  True  Airspeed  (TAS) 

3.  Groundspeed  (GS) 

4.  True  Course  (TC) 

5.  True  Heading  (TH) 

6.  Magnetic  Heading  (MH) 

7.  Altitude  (ALT) 

8.  Altitude  Deviation  (ALTDEV) 

9.  Roll  Rate  (ROLL) 
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10.  Vertical  Velocity  (WI) 

11.  Course  Deviaticm  (CD) 

12.  Latitude  (LAT) 

13.  Longitude  (LONG) 

14.  Tune 

15.  Switch  Hits/Keystrokes 

16.  Time  each  menu  page  displayed 

The  above  raw  data  were  put  into  a  congregate  database  and  various  statistical 
analyses  were  conducted  on  these  parameters  to  evaluate  the  measures  of  perftxmance 
listed  below. 

Measures  of  Performance 


1.  Control  Tlme/Tlme  Over  Steerpoint  Deviations:  Time  difference  in  minutes  and  seccmds 
from  (tesignated  control  times  (e.g.,  T.O.,  ARCT,  RZ  CT)  (Ref:  60-4,  Volume  DC) . 

2.  Control  Point/Steerpoint  Deviations:  Actual  distance  left  or  right  of  course  in  tenths  of 
miles  at  control  point  (e.g.,  ARCP,  RZ  PT)  (Ref:  SACR  60-4  Vol  DC). 

3.  Airspeed  Deviation:  Root  mean  square  of  deviation  from  planned  indicated  airspeed  for 
Air  Refueling  track  (ARCP  or  Precontact,  whichever  is  later,  until  5  minutes  before 
EAR)  (Ref:  SACR  60-4  Vol  DC). 

4.  Altitude  Deviation:  Total  deviation  from  flight  planned  altitude  +  ot  - 100  feet  in  total 
feet  (Ref:  SACR  60-4,  Vol  DC). 

5.  Weather  Deviation:  Minimum  distance  from  thunderstorm  in  tenths  of  miles 
(Ref:  AFR  60-16,  SAC  Sup  1). 

Measures  of  Workload 


*luhjective  Workload  Assessment  Technique  fSWATl 


SWAT  (Reid  et  al.,  1989)  provides  a  global  measure  of  subjective  workload. 
SWAT  is  divided  into  three  factors:  (1)  Time  load,  (2)  Mental  effect  load,  and  (3) 
Psychological  stress  load.  Time  load  depends  on  die- availability  of  spare  time  and  the 
overlap  of  task  activities.  It  is  rated  on  a  three-point  scale  from  1-Often  have  spare  time  to 
3- Aln^t  never  have  spare  time.  Mental  eff<»t  load  is  an  indicate  of  the  amount  of 
attention  or  mental  tasks  that  are  required  to  accomplish  a  task,  independent  of  the  number 
of  subtasks  or  time  limitations.  It  is  rated  on  a  three-point  scale  from  1-Veiy  little 
ctxiscious  mental  effent  or  concentration  required  to  3-Excessive  mental  effort  and 
c(»icentration  are  necessary.  Psychological  stress  load  refers  to  conditions  that  produce 
confusion,  frustration,  and/or  anxiety  during  task  performance  and,  therefore,  make  task 
acctxnplishment  seem  more  difficult  It  is  rated  on  a  three-point  scale  from  l-Little 
confusion,  risk,  frustration,  or  anxiety  exists  and  can  be  easily  accommodated,  to  3-High 
to  very  intense  stress  due  to  confusion,  frustration,  or  anxiety. 


The  SWAT  technique  was  used  to  determine  the  workload  of  each  crewmember 
throughout  each  of  the  missions.  Pilots  sorted  27  cards  with  various  descriptive  workload 
levels  of  the  three  factors  (Time,  Mental  effort,  and  Psychological  stress)  into  a  lowest  to 
highest  workload  arrangement  This  card  sort  ttilowed  die  in^vidual  to  show  which 
factors  were  considered  as  more  critical.  The  SWAT  card  sort  is  a  conjoint  measurement 
technique  that  is  subsequently  used  to  generate  a  lOO-pomt  interval  scale.  The  results  of 
the  data  can  then  be  submitted  fOT  traditional  statistical  analysis.  The  crew  members 
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performed  the  initial  card  sOTt  on  the  morning  of  the  first  day  and  immediately  after  the 
fourth  mission  on  the  last  day.  SWAT  ratings  were  requested  at  various  times  during  each 
flight  firom  each  of  the  pilots.  At  that  time,  &e  pilots  would  respond  with  a  numeric^ 
rating  (1, 2,  or  3)  for  each  of  the  factors.  For  example,  if  a  pilot  said  his  rating  was  132 
(one  th^  two),  it  meant  his  time  load  rating  was  one,  his  mental  effort  load  was  three,  and 
his  psychological  stress  load  was  two.  These  ratings  were  recorded  by  the  experimenter 
for  later  analysis. 


Subjective  WQRkload  Dominance  (SWORD) 


SWORD  (Vidulich,  1991)  uses  a  series  of  relative  judgments  comparing  the  relative 
workload  of  different  task  and  mission  segments  in  reference  to  die  aircraft  The  rater  was 
present  with  a  rating  sheet  that  listed  all  possible  paired  comparisons  of  the  tasks  (see 
example.  Figure  8).  ^e  task  appeared  on  the  left  side  of  the  line,  and  another  task 
appeared  on  the  right  The  rater  then  marked  the  "EQUAL”  slot  iJf  both  tasks  inflicted 
identical  levels  of  workload.  If  either  task  caus«l  higher  wcakload,  the  rater  mariced  a  slot 
closer  to  the  dcnninant  task.  The  greater  the  difference  between  the  two  tasks,  the  closer  the 
mark  was  placed  to  the  dominant  (higher  workload)  task. 

SWORD  EXAMPLE  SHEET 


Vary 

Abaoluta  Strong  Strang  Wook 

EQUAL 

Wank 

Vary 

strong  Strong  Abaoluta 

EXAMPLE  1  • 

Taaka  X  and  Y  ara  EQUAL 

In  WorkkMti 

X  _ 

— 

— 

Y 

EXAMPLE  2  • 

Taak  Y  causaa  a  llnia  mora 

Wortdoad. 

X  _ 

... 

✓  _ 

Y 

EXAMPLE  3  • 

Task  Y  eauaaa  a  lot  mora 

Workload. 

X  _ 

. 

>• 

1 

1 

1 

1 

1 

EXAMPLE  4  • 

Taak  X  eauaaa  aomawhat  mora 

Workload. 

X  _ 

✓ 

— 

— 

— 

Y 

Fi£ure  8.  Instructional  example  for  the  SWORD  rating  technique. 


Modified  Cooper-Harper  (MCH)  Scale 

The  MCH  scale  (Figure  9)  is  a  modified  version  of  the  original  Cooper-Harper 
scale  (Cooper  &  Harp^,  1969).  The  MCH  is  a  subjective  measure  of  workload.  The 
operate  enters  the  decision  tree  at  the  bottom  left  and  must  make  various  decisions  in 
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arriving  at  his  perceived  workload.  The  MCH  was  given  to  the  pilots  via  a  mission 
questi(Hinaire  to  determine  their  workload  for  various  task  requirements  and  again  at  the 
end  of  the  study  to  determine  their  workload  for  the  various  mission  segments  as  identified 
byWardetal.  (1991). 


Figure  9.  Decision  tree  for  the  Modified  Cooper-Harner  ratiny  scale 


Qucstipnnalrc  Data 

The  crews  were  required  to  answer  several  questionnaires  during  the  course  of  the  . 
experiment.  A  missitxi  specific  questionnaire  (Appendices  B-D)  was  actoodnistered  to  the 
crewmembers  immediately  following  the  mission.  This  questionnaire  was  used  to 
specifically  pinpoint  high  workload  areas  during  the  mission.  Additionally,  several 
identical  questions  were  asked  on  each  of  the  mission  specific  questionnaires  to  identify 
conunon  high  workload  areas  and  also  to  serve  as  a  mission  difficulty  manipulation  check. 
On  the  final  day  of  the  study  the  crews  were  required  to  complete  a  39-page  system 
questionnaire  (Appendix  E).  The  questionnaire  had  152  multiple  choice  items;  each  of 
which  had  a  comments  section  allowing  the  crewmember  to  fully  explain  his  answer.  This 
questitxinaire  was  designed  to  identify  specific  mission  functional  requirements  necessary 
for  successful  mission  accomplishment. 
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RESULTS  AND  DISCUSSION 

This  results  section  is  broken  up  into  four  main  sections.  The  first  section  analyzes 
the  objective  performance  measures  in  order  to  evaluate  whether  the  conceptual  cockpit 
design  would  have  met  current  Federal  Aviation  Agency,  Air  Force,  and  Strategic  Av 
Command  regulations  and  directives  concerning  mission  perfomiance.  The  second  section. 
Mission  Difficulty,  explains  the  validity  of  our  mission  difficulty  manipulations.  The  third 
section  looks  at  the  various  subjective  measures  used  in  this  study  to  determine  the 
feasibility  of  the  conceptual  cockpit  desi^  and  potential  areas  of  concern.  Finally,  the 
system  questionnaire  data  are  discussed  in  detail.  Each  of  the  sections  overlap  to  varying 
degrros  as  many  of  the  measures  allowed  the  authors  to  draw  conclusions  to  different 
questions.  The  first  three  sections  address  the  question,  "Is  it  feasible  to  eliminate  the 
navigator  from  the  KC-13S  cockpit"  The  final  section  identifies  the  minimum  functional 
requirements  necessary  for  the  elimination  of  the  navigator  to  be  feasible. 

Throughout  the  four  sections,  the  reader  will  be  presented  multiple  figures  fix’  both 
the  KC-135  results  and  the  KC-10  results.  SWAT  and  SWORD  figures  represent  the 
overall  group  rating,  unless  otherwise  stated.  Figures  whose  title  tegin  with  "KC-135" 
represent  the  overall  group  rating  for  both  the  KC-13S  pilots  and  copilots  combined. 

Figure  titles  beginning  with  "KC-10"  represent  the  ove^  rating  fix  both  the  KC-10  pilots 
and  copilots  combing.  The  KC-135  and  KC-10  sample  sizes  were  20  and  4,  respectively. 

The  various  groups  of  interest  are  noted  in  the  legend  at  the  top  of  each  figure.  The 
"Pilot"  group  refers  to  KC-135  pilots/aircraft  commanders  only.  The  "Copilot"  and 
"Navigator"  groups  refer  to  the  KC-135  copilots  and  navigators,  respectively.  The  "KC- 
10"  group  include  both  KC-10  pilots  and  copilots.  The  sample  size  (n)  for  each  group, 
unless  specifically  noted  otherwise,  follows:  Pilots  (n=10);  O)pilots  (n=10);  Navigatws 
(n=10);  and  KC-10  (n»4).  Consequently,  statistical  analysis  of  the  KC-10  data  was  not 
possible.  Rather,  KC-10  data  are  presented  as  a  supplement  to  the  KC-135  results  and 
provide  valuable  trend  information. 

Objective  Measures 

Data  collected  on  a  1-Hz  cycle  for  each  of  the  three  missions  were  reduced  and 
organized  into  five  Measures  of  Performance  categories:  (1)  Control  Time  Over  Steerpoint 
Deviations,  (2)  Control  Point/Steapoint  Deviations,  {3)  Airspeed  Deviation  (4)  Altitude 
Deviation,  a^  (5)Weather  Deviation.  These  categtxies  were  then  evaluated  against 
Strategic  Air  Command  Regulations  SACR  60-4,  Vol.  I;  SACR  60-4,  Vol.  IV;  and  Air 
Force  Regulation  AFR  60-16,  SAC  Supplement  1.  Additionally,  recorded  comments  from 
the  navigator  observers  and  the  experimenters  were  used  to  supplement  the  data.  The 
results  of  such  an  analysis  are  presented  by  category. 

Control  Time  Over  Steerpoint  Deviations 

The  control  time  over  steerpoint  deviation  was  evaluated  against  the  actual 
scheduled  Rendezvous  (RZ)  time/Air  Refueling  Control  Time  (lime)  for  each  mission. 
SACR  60-4,  Vol.  I  dictates  that  all  timing  control  points  be  made  wiAin  +/-3  minutes.  A 
review  of  the  recorded  data  indicated  that  all  actual  control  times  (a  total  of  34)  met  SACR 
60-4  ^uiiements.  No  control  time  difficulties  were  recognized  by  the  observer  or  the 
experimenter. 
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Control  Point/Steerpoint  Deviations 

Course  deviations  are  not  to  exceed  10  NM  on  either  side  of  track  in  accordance 
with  SAC^  60-4,  Vol.  IV.  Course  deviation  was  evaluated  throughout  flight  with 
increased  emphasis  placed  on  the  period  5  minutes  before  air  refueling  to  10  minutes  after 
rendezvous  completion.  No  course  deviation  exceeded  10  NM  from  track  with  the 
exception  of  those  required  to  avoid  thunderstorms.  AU  of  these  deviations  were 
coordinated  and  approved  by  Center.  No  course  deviations  beyond  ten  miles  were 
identified  by  either  the  navigator  observer  or  the  experimenter. 

Airspeed  Deviation 

Since  airspeed  is  a  primary  method  for  time  control,  airspeed  deviation  was 
evaluat»i  during  &e  air  refueling  portion  of  flight  It  is  during  this  time  period  that  the 
tanker  pilots  must  maintain  a  set  airspeed.  SACR  60-4,  Vol.  W  states  airspeed  must  be 
within  -I-/-10  knots  for  full  qualification.  In  a  review  of  the  data,  less  than  1%  of  all  data 
points  exceeded  10  knots.  Of  the  data  points  that  (hd  not  meet  the  criteria,  all  were 
associated  with  the  avoidance  of  the  weather  and  the  effort  to  maintain  end  air  refueling 
timing  considered  as  acceptable  deviations. 

Altitude  Deviation 

SACR  60-4,  Vol  rv  requires  that  aircraft  altitude  be  maintained  within  +/-150  feet 
of  assigned  altitude.  Since  crewmembers  can  and  often  did  request  altitudes  other  than 
those  originally  planned,  increased  importance  was  placed  upon  the  observer  and 
experimenter  notes  to  derive  actual  altitude  deviations.  Additionally,  altitude  during  the  air 
refrieling  segment  of  the  mission  (which  was  consistent  across  crews)  was  used  as  baseUne 
for  determining  altitude  deviations.  In  a  review  of  the  data,  only  three  altitude  deviations 
exceeded  the  150  feet  criteria.  Only  one  of  the  three  deviations  exceeded  500  feet  All  of 
the  deviations  were  momentary  and  corrective  action  was  initiated  by  the  pilots  without 
outside  intervention. 


Weather  Deviation 

APR  60-16,  SAC  Sup  1  imposes  minimum  distance  criteria  for  thunderstorm 
avoidance.  Specifically,  thunderstorms  must  be  avoided  by  20  NM  at  or  above  FL  230  and 
by  10  NM  below  FL  230.  Thunderstorm  avoidance  was  evaluated  throughout  the  entire 
mission.  No  crew  had  any  difficulty  in  avoiding  thunderstorm  by  the  prescribed  distance. 

Of  the  five  performance  measures,  only  the  altitude  deviation  category  resulted  in 
less  than  qualified  activity,  although  momentary  in  nature.  Crews  indicate  that  an  altitude 
warning  signal  that  notifies  a  crew  when  they  have  gone  beyond  an  assigned  altitude  (a  role 
previously  performed  by  the  navigator)  would  ail  but  eliminate  this  potential  problem.  The 
above  performance  measure  results  indicate  that  a  two-person  (no-nav)  conceptual  cockpit 
design  can  result  in  qualified  activity  and  successful  mission  performance. 

Mission  Difficulty 

The  three  missions  (Minot,  Mather,  and  Castle)  were  planned  with  varying  degrees 
of  difficulty  based  on  the  following  factors;  (1)  Takeoff  time,  (2)  Cell  procedures,  (3) 
Inflight  replanning,  (4)  Weather,  and  (5)  System/equipment  malfunctions.  The  various 
mission  difficulties  were:  (1)  Easy  (Minot),  (2)  Medium  (Mather),  and  (3)  Hard  (Castle). 
Figure  10  shows  the  SWAT  ratings  provided  by  the  KC-135  crews.  Figure  1 1  is  the 
SWAT  ratings  graph  for  the  KC- 10  aircrews. 
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The  x-axis  lists  the  missions  flown  by  each  of  the  KC-135  aircrews  (Figure  10)  and 
KC-10  aircrews  (Figure  1 1).  The  y-axis  represents  an  interval  scale  indicating  the  overall 
rating  of  the  group  for  each  of  the  missions.  The  higher  the  rating,  the  higher  the  perceived 
wOTldoad.  The  dashed  line  between  points  is  provided  as  a  visual  aid  in  determining  trend 
information  between  missions.  As  seen  in  Hgures  10  and  1 1,  mission  difficulty  as 
indicated  by  overall  SWAT  woiidoad  did  increase  from  the  Minot  to  the  Mather  mission 
and  from  the  Mather  missitm  to  the  Castle  mission  as  predicted.  This  effect  was 
statistically  significant,  £(2, 51)  =  7.29,  o<.01. 


Additionally,  crewmembers  were  asked  to  choose  the  adjective  (easy,  medium, 
hard)  that  best  describes  the  difficulty  of  the  mission  just  flown.  The  adjectives  were  later 
assigned  a  value  of  one  for  easy,  two  for  medium,  and  three  fw  hard.  The  subject  group 
for  Figure  12  consisted  of  all  the  KC-13S  pilots,  cq)ilots,  and  navigators  for  each  of  the 
three  missions  (a=90).  As  seen  in  Figure  12,  the  missions  were  increasingly  more  difficult 
in  the  hypothesized  directicxi.  A  two-way  (mission  x  position)  analysis  of  variance 
(ANOVA)  of  these  responses  resulted  in  a  significant  main  effect  for  mission 
(E(2, 87)  =  35.07,  ii<.01),  but  not  for  position.  There  was  no  interaction.  This  indicates 
that  mission  difficulty  was  the  result  of  the  mission  manipulations  and  were  in  the 
hypothesized  direction. 
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Figure  12.  Adjective  workload  ratine  for  each  mission. 

Crewmembers  were  also  asked  to  rate  e^h  of  the  missions  using  the  Modified 
Cooper-Harper  (MCH)  scale  (Figxue  9).  Figure  13  shows  the  results  of  that  rating 
process.  The  x-axis  lists  the  three  missions  flown  by  the  KC-135  aircrews.  The  y-axis 
represents  an  interval  scale  of  the  mean  rating  for  that  mission.  A  rating  of  1  represents 
little  workload;  whereas,  a  rating  of  10  indicates  an  excessive  workload  level.  The  figure 
shows  the  Minot  and  Mather  missions  were  not  different;  but  both  were  significantly 
different  from  the  Castle  mission,  F(2, 83)  =  7.79,  jj<.01.  Additionally,  a  two-way 
ANOVA  resulted  in  a  significant  main  effect  (E(2, 83)  =  7.79,  p<.01)  for  mission 
difficulty,  but  there  was  no  main  effect  for  crew  position  and  there  was  no  interaction 
effect.  This  again  indicates  mission  difficulty  was  the  result  of  the  mission  manipulations. 
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Fiyurg  13.  Modified  Coongr.Harngr  workload  ratiny  for  each  mission. 


The  general  trend  die  data  follows  the  expected  patt^  of  results.  Mission 
difficulty  increased  firom  the  Minot  mission  to  the  B^dier  mission  to  the  Castle  missicxi. 
However,  the  increase  in  difficulty  between  the  Minot  and  Mather  missions  was  not  always 
significant  and,  as  shown  in  Figure  13,  was  sometinies  nuniinaL  This  is  a  consequence  of 
a  last  minute  mission  profile  change.  The  Minot  mission  was  originally  planned  with  no 
hydraulic  malfunction,  but  in  order  to  evaluate  the  potoitial  woridoad  increase  of  such  an 
event,  it  was  added  on.  The  Mather  mission  was  originally  considered  for  such  an  event, 
but  it  was  felt  that  such  an  event  would  have  increased  the  workload  level  to  that  of  the 
Castle  mission. 

Given  a  preliminaiy  analysis  indicated  the  Minot  and  Madier  missions  were  not 
significantly  different  from  each  other  on  the  MCH,  KC>10  crewmembers  were  only 
required  to  fly  the  Mather  and  Castle  missions  for  the  data  collection  portimi  of  die 
experiment  The  Mather  mission  was  chosen  over  the  Minot  mission  as  it  was  the  only 
mission  requiring  the  crew  to  perform  a  point  parallel  rendezvous.  This  allowed  for  a 
comparison  of  the  KC-13S  and  KC-10  data  for  both  types  of  the  air  refueling  rendezvous. 

In  summary,  the  increase  in  mission  difficulty  from  the  Minot  mission  to  the  Mather 
mission  to  the  Casde  mission  was  as  exposed  and  proved  the  manipulatitxi  of  the  mission 
difficulty  was  successful.  Additionally,  the  missions  flown  repesented  a  wide  range  of 
workloads  aixi  were  representative  of  current  operational  missions  flown  today. 
Accordingly,  the  confi^nce  in  which  we  make  die  following  conclusions  is  g^dy 
increased. 


SWAT 


As  mentioned  previously,  SWAT  provides  a  subjective  global  assessment  of 
woridoad.  A  review  of  Figure  10  shows  die  crewmembi^  did  experience  increasing 
mission  difficulty  as  indicated  by  their  overall  SWAT  rating  for  each  of  the  missions.  This 
provided  the  ba^  for  determining  the  validity  of  our  missitxi  difficult  manipulatitxis. 
SWAT  can  also  provide  the  basis  for  identif^g  specific  events  during  the  mission  and 
their  assodated  wcnkloads.  This  was  accomplishi^  by  identifying  sp«nfic  events  (events 
bdieved  to  be  associated  with  higher  workloads)  within  each  of  die  missions  prior  to  die 
study.  Then,  at  designated  times  during  the  mission,  the  experimenter  requested  a  SWAT 
scrae  for  each  of  die  events.  These  scores  were  then  placed  into  the  database  and  wcvkload 
scores  were  determined.  These  workload  scores  were  then  averaged  and  the  resulting 
graphs  were  developed. 

Figures  14-18  allow  for  the  analysis  of  each  mission  for  each  of  the  specific  events. 
Unlike  Figures  10  and  1 1,  the  x-axis  lists  the  specific  mission  events  radwr  than  the 
missions  flown.  The  y-axis  is  the  same  as  Hgures  10  and  1 1  and  represents  the  mean 
rating  of  the  group  for  each  of  the  mission  events.  The  higher  the  rating,  the  higher  the 
perceived  wt^oad.  The  dashed  line  between  pdnts  is  again  only  provitkd  as  a  visual  aid 
in  determining  trend  information  between  die  missions  events.  Reid  and  his  associates 
(G.  B.  Reid,  personal  communication,  March  4, 1991)  at  the  Armstrong  Laboatory  are 
currendy  womng  on  determining  the  SWAT  ssxte  for  woikloads  associated  with  moderate 
effort  levels  (areas  of  oxicem).  Initial  results  indicate  that  such  a  score  will  fall  between 
40  and  50.  Therefore,  for  purposes  of  this  study,  events  with  a  SWAT  workload  rating 
above  40  are  identified  as  an  area  of  concern. 

Figure  14  presents  the  SWAT  ratings  for  the  three  Minot  mission  events:  (1)  Late 
takeoff,  (2)  Air  refueling,  and  (3)  Hydraulic  malfunction.  Figures  IS  and  16  present  the 
the  KC-13S  crew  and  the  KC- 10  crew  SWAT  ratings  for  the  three  Mather  mission  events: 
(1)  Late  takeoffCell,  (2)  WX  avoidance,  and  (3)  Replan.  Figures  17  and  18  are  the 
respective  SWAT  ratings  for  the  Casde  mission  events:  (1)  &11,  (2)  Rqilan,  (3)  Random 
Refuel,  and  (4)  WX  Divert/Missed  App. 

Lau  takeoff  cowers  the  late  takeoff  the  crew  encountered,  coordination  with  Center, 
and  the  inflight  replanning  needed  to  regain  scheduled  mission  timing.  Cell  events  included 
the  cell  departure  and  join-up  phase  of  flight,  cell  position  changes,  ^  associated 
communications.  The  cur  rffuelingirandom  refuel  evoit  included  the  on-course  or  point 
parallel  rendezvous,  fuel  transfer  via  the  fuel  management  panel,  the  termination  of  the 
refueling,  and  all  associated  radio  communications.  WX  ovotdnnce  required  the 
identification  and  avoidance  of  thunderstorms,  in  addition  to  Center  coordination  of  any 
route  deviations/changes.  Replan  events  covered  navigation  route  changes,  timing 
changes,  and  Uk  associated  coordination  with  Center.  VMhydraulkrr^fUnctioneweni 
covered  the  malfunction,  all  communications,  and  the  approach  and  landing.  The  Wx 
divert  Imissed  approach  event  was  based  on  Ae  tire  crew's  approach  into  C^tle,  the  missed 
approach  procedure,  inflight  replanning  to  Beale,  and  all  associated  ctxnmunications. 


A  review  of  Flgiire  14  reveals  that  none  of  the  Minot  mission  events  received  a  rating 
over  40  by  the  KC-135  pilots/copilots.  This  indicates  an  "easy "  mission  such  as  this  one 
would  result  in  workload  that  are  manageable,  given  the  CS^  ctmceptual  cockpit  design. 
Similarly,  the  KC-13S  pilots/copilots  consider^  the  workloads  for  the  Mather  mission 
events  (late  takeofObell  weather  avoidance  and  replan)  as  manageable,  although  slightly 
higher  than  the  Minot  events  (Figure  15).  Mintx  mean  rating  was  25.29  and  the  Mather 
mean  rating  equalled  32.68.  KC-10  crews  reported  woiidoads  associated  with  the  Mather 
mission  events  as  very  low  (Figure  16),  despite  the  fact  that  they  were  not  qualified  in  the 
KC-135.  This  was  attributed  to  differences  in  experience  and  training.  Specifically,  KC- 
10  pilots/ct^ilots  are  trained  in  navigation  and  ra^  procedures.  They  are  the  navigator  on 
the  aircraft  and  currently  fly  with  a  cockpit  similar  to  the  CSEF  conceptual  cockpit  design. 
The  results  presented  in  Figure  16  may  represent  the  effects  of  increa^  training. 
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Figure  17  reveals  three  of  the  four  Castle  events  as  potential  areas  of  concern  for  KC- 
135  pilots.  The  replan,  random  refuel,  and  WX  divert/missed  approach  events  all  had 
SWAT  ratings  exceeding  the  40  chosen  as  our  area  of  concern.  TTiis  mission  was 
intentionally  designed  to  drive  workloads  above  the  redline  value  of  40  in  order  to  evaluate 
potential  system  design  problems  (discussed  in  the  questionnaire  section).  In  discussions 
and  inquiries  into  the  difficulties  involved  with  these  areas  of  flight,  KC-13S  pilots/copilots 
felt  training  in  navigation  procedures  was  insufficient  They  also  felt  that  the  requirement 
to  fly  the  airplane  f^l  dme  due  to  the  autopilot  failure  was  definitely  a  workload  producer. 

In  contrast  KC-10  pilots/copilots  did  not  experience  the  same  workloads  as  that 
experienced  by  the  KC-13S  crewmembers.  Due  to  inadequate  KC-10  sample  size, 
statistical  testing  was  not  feasible.  Figure  18  shows  the  KC-10  crewmembers'  workload 
rating  for  the  th^  areas  of  concern  were  all  under  the  hypothetical  "red  line"  of  40.  This  is 
attributed  to  a  large  degree  to  the  increased  training  the  KC-10  crewmembers  receive  in 
similar  flight  systems  and  navigational  procedures.  KC-10  crewmembers  stated  that  their 
workloads  were  more  the  result  of  having  to  fly  the  plane  continuously  throughout  flight 
and  being  unqualified  in  the  plane. 

As  a  result  of  the  SWAT  global  assessment  of  the  crewmembers  workloads,  further 
explanation  as  to  what  specifically  was  the  source  of  the  higher  workloads  associated  with 
the  three  areas  of  concern  was  needed.  Identification  of  the  specific  tasks  causing  higher 
WOTkloads  would  aid  engineers  in  developing  the  necessary  functional  requirements  for 
successful  mission  accomplishment  The  ability  to  identify  specific  tasks  within  the 
potential  areas  of  concern  was  the  reason  for  collecting  SWORD  data  during  the 
experiment 


SWORD 

The  Subjective  Workload  Dominance  technique  uses  a  series  of  relative  judgments 
comparing  the  relative  workload  of  different  tasks  and  mission  segments  in  reference  to  the 
airci^  Such  a  comparison  provides  valuable  information  as  to  whether  one  task  is 
believed  to  cause  more  workload  than  another  task.  It  also  provides  an  indication  of  which 
system  (aircraft  vs.  simulator)  resulted  in  higher  degrees  of  workload. 

SWORD  data  were  coUeaed  for  three  mission  segments  for  each  of  the  three 
missions.  The  nussion  segments  chosen  for  each  of  the  Minot,  Mather,  and  C^tle 
missions  were  determined  prior  to  data  collection,  based  on  those  three  segments 
hypothesized  to  be  associated  with  higher  worklt^  levels.  The  tasks  selected  for  a  given 
mission  segment  were  based  on  (1)  tasks  believed  to  be  performed  most  often  and 
(2)  the  task  anticipated  level  of  difficulty.  These  determinations  were  a  direct  result  of  the 
function  analysis  previously  performed  by  Ward  et  al.  (1991). 

To  aid  in  the  understanding  of  the  results,  a  detailed  explanation  of  Figure  19  is 
provided.  Figure  19  presents  the  results  from  the  cruise  segment  of  the  Minot  mission. 
The  x-axis  represents  the  particular  tasks  of  interest  for  the  chosen  mission  seginent  The 
y-axis  is  an  interval  scale  used  to  indicate  the  relative  relationship  between  the  aircraft  and 
the  simulamr  for  a  given  task.  The  asterisk  indicates  the  relative  workload  associated  with 
a  given  task  in  the  simulate  (i.e.,  CSEF  conceptual  cockpit  design  with  no  navigator);  the 
square  inthcates  the  aircraft  (i.e.,  current  model  aircraft  with  a  navigator).  The  lines 
between  asteri^s  and  between  squares  provide  trend  information.  Subsequent  figures 
(Figures  2(>'26)  are  presented  using  the  same  scale  graph.  The  particular  mission,  mission 
segment,  airert^  and  tasks  vary  from  one  figure  to  the  next.  Additionally,  similar  figures 
for  the  KC-10  crewmembers  are  directly  below  those  of  the  KC-135  to  aid  in  the 
ctxnparison  between  the  two  aircraft 
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Figure  19.  Minot  SWORD  ratines  for  the  cruise  segment. 

Figure  19  provides  the  results  for  the  cruise  segment  of  the  Minot  mission.  The  three 
tasks  compared  during  this  segment  were  the  flying  task,  the  communications  task,  and  the 
navigation  task.  The  flying  task  was  based  on  the  level  of  difficulty  fa-  maintaining 
airspeed,  altitude,  and  heading.  The  communications  task  involved  all  communications 
with  Center  and  the  receiver  aircraft,  the  selection  of  assigned  radio  frequencies,  and  the 
control  of  the  UHF  and  HF  radios.  The  navigation  task  involved  all  activities  to  ensure 
route  timing  and  course  control  were  met. 

A  review  of  Figure  19  reveals  the  cruise  segment  was  beUeved  to  be  lower  in 
wtRidoad  for  the  CSEF  conceptual  cockpit  design  dian  for  the  current  aircraft,  althou^  not 
statistically  significant.  Only  the  flying  task  was  rated  slightly  higher  for  the  simulatOT  than 
for  the  aircraft,  although  the  two  were  not  statistically  different  from  each  other.  These 
results  indicate  that,  for  the  cruise  portion  of  flight,  the  CSEF  conceptual  cockpit  design 
resulted  in  wOTkloa^  that  were  not  any  higher  than  that  of  the  current  aircraft 

Figure  20  indicates  that  except  for  the  flying  task  (Maintaining  LOC/GS),  the 
simulator  was  again  rated  as  lower  (but  not  significandy)  in  worklot^  relative  to  the 
crewmember's  current  aircraft  fcH*  the  approach  and  lanc^g  segment  of  the  Minot  mission. 
As  before,  the  simulator  was  rated  as  slighdy  higher  in  wc^oad  for  the  flying  task,  but 
once  again  was  not  statistically  different  from  that  of  the  aircraft.  A  trend  of  higher 
woricl^tds  for  flying  the  simulate  is  already  devious  at  this  point  The  remaining  figures 
(Figures  21-26)  also  indicate  that  flying  the  simulate  was  rated  as  more  difficult  (hi^ier 
workload)  than  flying  the  reference  aircraft  for  both  the  KC-135  and  KC-10  pilots.  When 
questioned  why  they  felt  the  simulator  was  harder  to  fly,  the  crewmembers  responded  that 
"all "  simulators  are  harder  to  fly  than  the  actual  aircraft  They  stated  the  control  loads  and 
trim  capabilities  of  this  and  other  simulators  are  not  as  realistic  as  the  actual  plane  and, 
therefore,  increase  the  difficulty  of  keeping  the  aircraft  propeily  trimmed  for  level  fli^t 
However,  when  asked  how  this  simulator  compared  to  other  training  simulators,  the  pilots 
felt  the  simulator  was  a  good  simulator.  One  subject  even  stated  that  it  was  the  test  (most 
realistic)  KC-13S  simulator  he  had  ever  flown. 
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Figures  21  and  22  are  the  graphed  results  of  the  Air  Refueling  segment  fc»r  each 
mission.  The  fueling  (FUEL)  t^  involved  the  management  of  the  Fuel  Management 
Panel  (FMP),  transfer  of  fuel,  maintenance  of  the  proper  center  of  gravity  (Cg),  and  the 
mviitoring  of  fuel  flows.  The  ccmimunicaticms  ta^  and  the  flying  tasks  were  similar  to 
thosv'^  ?xplaiiM»l  previously.  The  KC-135  graph  (Figure  21)  stows  that  only  the  flying 
portion  of  the  air  refueling  segimnt  was  considered  as  more  difficult  in  the  simulator 
£(2, 1 17)  s  7.94,  p<.01.  The  pilots  again  attributed  this  increase  to  the  idea  that  all 
simulators  are  more  (tifficult  to  fly  than  an  actual  aircraft  This  effect  was  even  man 
noticeable  for  the  Castle  mission,  i(38)  =  3.8153,  ^01.  This  is  the  result  of  the  increased 
"handsKHi”  flying  time  that  resulted  fircra  the  autopilot  failure  experienced  during  the  Castle 
mission.  No  sig^cant  effects  were  found  between  the  reference  aircraft  and  the  simulator 
for  die  communicadons  or  fueling  task. 

Figure  22  reveals  KC-10  pilots  reported  higher  workloads  in  the  simulator  than  the 
aircraft  for  each  task  during  the  air  refueling  portion  of  the  missions.  An  initial  reaction  to 
this  situaticm  might  be  to  dnd  the  conceptud  cockpit  design  as  flawed,  but  further 
investigation  uncovered  this  apparent  discrepancy  to  be  the  result  of  the  differences 
between  the  two  aircraft.  Only  the  fueling  and  flyiiig  tasks  resulted  in  differences  greater 
than  0.1,  indicating  the  tasks  might  be  significant  given  a  larger  sample  size.  The  flying 
difficulties  are  due  to  two  reasons.  The  first  reason  is  that  the  pilots  were  not  qualified  in 
the  KC*13S  and,  therefore,  their  reference  aircraft  (KC-10)  would  be,  by  default,  easier  to 
fly.  The  second  is  again  attributed  to  the  increased  difficulty  associated  vdth  flying  a 
simulate  as  compared  to  an  actual  aircraft.  Tto  fueling  task  may  be  the  result  of  Ae  KC- 
10  crew  duties.  Ilie  KC-10  has  a  flight  engineer  onboard  the  aircraft  who  is  responsible 
for  the  fuel  management  and  the  weight  and  balance  of  the  aircraft.  KC-10  pilots  are  not 
responsible  for  any  of  the  fueling  task  (as  previously  defined)  on  board  the  KC-10. 
Accordingly,  the  ^el  tasking  rating  was  extremely  low  (0.03)  for  the  aircraft  while  the  fuel 
rating  for  the  simulator  was  substantially  higher  (1. 6-2.0).  An  inspection  of  the 
ctxnn  mication  tasks  shows  slight  differences  in  woridcod  between  the  simulator  and  the 
aircrari  for  the  Mather  mission  and  no  difference  in  workload  for  the  Castle.  This  indicates 
tlie  communication  task  does  not  substantially  increase  workload  during  the  air  refueling 
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Figure  21.  KC-13S  SWORD  ratines  for  the  air  refueling  segment. 
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Figure  22.  KC-IO  SWORD  ratines  for  the  air  refueling  segment. 
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segment  of  flight  The  reader  must  also  remember  that  KC- 10  data  are  based  on  a  sample 
size  of  four.  Such  a  small  sample  size  for  the  KC-10  data  makes  statistical  significance 
inq)robabie.  However,  the  trend  information  provides  valuable  information  concerning 
potential  areas  of  concern. 

Figures  23  and  24  are  the  results  of  the  cell  departure  and  join-up  segment  of  the 
Mather  and  Castle  missions.  The  radar  task  involv^  the  tuning  of  the  radar  set  radar 
scope  interpretatitm,  and  scale  selection  to  effectively  perform  a  cell  departure  and  join-up 
via  radar  means.  The  other  two  tasks  were  explained  previously.  Once  again,  only  the 
flying  task  was  rated  as  significantly  higher  workload  in  the  simulator  than  in  the  aircraft 
F(2, 38)  =  3.98,  p<.01,  but  only  for  the  Castle  mission.  This  may  be  attributed  to  the 
autopilot  failure,  which  required  an  increase  in  manual  flight  by  the  crew.  The  remaining 
comparisons  resulted  in  no  significant  differences  between  the  aircraft  and  the  simulator. 
This  further  supports  the  idea  that  this  conceptual  cockpit  design  did  not  significantly 
increase  mission  wcnrkload. 

Figures  25  and  26  indicate  the  workload  ratings  for  the  weather  divert  portion  of 
flight  Weather  divert  segment  of  the  mission  involved  the  avoidance  of  thunderstorms 
throughout  the  course  and  the  ensuing  coordination  with  Center.  The  storm  avoidance  task 
evaluated  the  capability  to  identify  the  weather  on  radar  and  the  tuning  of  the  radar.  The 
inflight  replan  task  required  the  pilot  to  determine  his  best  route  of  flight  to  avoid  the 
thunderstorm  and  the  use  of  the  mission  management  system  to  make  that  determination. 
The  communications  task  was  described  earlier.  For  both  the  KC-135  and  the  KC-10,  the 
pilots  generally  rated  workload  during  the  weather  divert  segment  of  the  mission  as  lower 
for  the  simulator  than  for  the  aircraft. 

Generally  speaking,  the  SWORD  results  indicate  workload  was  not  any  higher  for  the 
simulator  than  for  the  akcraft  The  two  excepticms  to  this  statement  were  the  actual  flying 
capabilities  of  the  simulator  and  the  air  refueling  segment  for  the  KC-10  pilots.  The  pUots 
unanitrxiusly  stated  that  the  major  workload  inducer  during  the  flying  task  was  the  fact  the 
simulator  could  not  be  trimmed  as  well  as  the  aircraft  The  increased  workload 
encountered  by  the  KC-10  pilots  for  the  fueling  task  during  the  air  refueling  segment  may 
be  the  direct  result  of  the  lack  of  a  flight  engineer.  Since  a  flight  engineer  is  responsible  for 
the  fuel  task  in  the  KC-10,  it  follows  the  KOlO  pilots  would  see  their  performance  on  this 
task  in  the  simulator  to  be  higher  workload  than  &e  aircraft.  The  most  important  point  to 
be  made  about  the  SWORD  results  is  that  relative  workloads  in  the  conceptual  coclqnt 
designed  simulator  were  not  higher  than  those  cunently  believed  to  occur  in  the  airoraft, 
despite  the  fact  that  the  navigator  was  eliminated  from  the  crew.  This  finding  by  itself  is 
significant,  since  its  supports  the  feasibility  of  such  an  endeavor. 
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Figure  23.  KC.135  SWORD  ratings  for  the  cell  departure  and  join.up  seymenL 
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Figure  24.  KC-IO  SWORD  ratings  for  the  cell  departure  and  join-up  segment. 
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Figure  25.  KC-13S  SWORD  ratings  for  the  weather  divert  segment. 
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Figure  26.  KC-IO  SWORD  ratings  for  the  weather  divert  segment. 
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The  Modified  Cooper  Harper  scale  (Figure  9)  was  used  in  conjunction  with  the 
mission  specific  questionnaires  to  evaluate  specific  workloads  associated  with  five  different 
tasks  common  to  all  missions:  (1)  Air  refueling,  (2)  Communications,  (3)  Navigation,  (4) 
Paperwork,  and  (5)  Piloting.  A  review  of  figures  13  and  27  shows  only  Ae  (Tastle  mission 
as  reaching  or  approaching  a  workload  level  of  four,  the  level  consider^  by  CSEF 
personnel  as  the  first  level  of  concern.  Below  four,  tasks  were  considered  to  be  easily 
managed  and  of  little  concern. 

Figures  28-30  represent  the  MCH  results  for  the  Minot,  Mather,  and  Castle  missitms, 
respectively.  Hie  x-axis  lists  the  particular  task.  The  y-axis  is  the  MCH  numerical  rating 
scde.  The  higher  the  numerical  rating,  the  hij^er  the  p^eived  workload.  The  Minot 
missicm  provides  the  ratings  for  the  KC-13S  pilots,  copilots,  and  navigators.  The  Mather 
and  Casde  Missions  provide  additional  ratings  for  the  KC-10  pilots.  Figure  28,  the  Minot 
mission,  indicates  that  no  task  was  consider^  to  be  in  a  region  of  concern.  The  Mather 
mission  (Figure  29)  indicates  that  only  the  aerial  refueling  tasks  were  considered  to  be  in  an 
area  of  concern  and  then,  only  by  the  navigators.  Castle  mission  results  (Figure  30)  were 
similariy  expressed.  The  navigators  identified  aerial  refueling,  navigation,  and  piloting 
tasks  as  tasks  with  increased  workload  and,  therefore,  of  concern.  However,  with  the 
excepti(Mi  of  the  piloting  task,  no  KC-13S  pilot  task  rating  was  at  or  above  four. 

Several  trends  emerge  from  Figures  28-30.  First,  with  one  exception  (Mather  piloting 
task),  navigators  always  rated  the  task  workload  in  question  highest  among  the 
crewmemtos.  Second,  with  the  exception  of  die  piloting  task,  the  KC-10  pilots  always 
rated  the  task  in  question  as  lowest  Third,  generally  speaking,  the  crews  felt  that  all  of  the 
tasks  were  manageable.  Each  of  these  trends  is  explainable. 

Since  the  navigators  were  only  observers  and  cannot  entirely  tap  into  the  internal 
stress  levels  that  each  of  die  pilots  was  experiencing,  it  is  possible  that  their  ratings  may  be 
biased.  Additionally,  the  navigators  had  a  vested  interest  in  the  project  Specifically,  if  the 
study  demonstrated  the  feasibility  of  the  design,  the  navigators  might  be  out  of  a  job.  This 
ccxicem  was  repressed  to  the  experimenters  several  times.  The  KC-10  ratings  reflect  what 
might  be  considered  training  effects.  Since  the  KC-10  crews  fly  a  similar  type  of  system, 
are  trained  in  radar  procedures  and  navigation  techniques,  and  currently  fly  with  no 
navigator,  they  were  much  more  efficient  in  those  procedures  fenr  which  the  KC-13S  pilots 
were  minimally  qualified.  The  third  trend  exhibited  in  the  piloting  task  was  in  large  part 
attributed  to  the  difficulty  in  flying  a  simulator  instead  of  an  actual  aircraft,  as  explained 
earlier.  Given  the  explanations  above,  the  navigates'  ratings  were  considered  as  the 
maximum  workload  that  would  probably  occur  in  flight;  whereas,  the  KC- 10  workload 
ratings  were  identified  as  the  minimal  level  of  workl^  associated  with  each  of  the  tasks. 
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A  final  question  identifying  tlw  peicent^e  that  each  task  was  believed  to  be  performed 
is  presented  in  Figure  31.  Tte  x-axis  again  li^  the  same  tasks  as  the  MCH  workload  scale 
above.  However,  tiie  y-axis  is  based  on  the  relative  percentage  of  time  that  each  task  was 
believed  to  be  p^onned.  Percentage  of  tasks  across  the  missitxis  remained  relatively 
stable  as  shown  in  Figure  31.  Only  4  percentage  points  separated  percentage  estimates  in 
the  worst  case  (AIR  REF).  However,  as  seen  in  Hgure  32,  task  percentages  varied 
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dramatically  between  the  crew  positions  (Minot  and  Mather  mission  resulted  in  similar 
results,  but  are  not  presented).  SpeciHcally,  the  pilots  were  involved  much  more  with  the 
piloting  tasks;  whereas,  the  copilots  were  much  more  involved  with  the  navigation  tasks. 
Since  crews  were  not  told  who  was  to  perform  what  tasks,  this  percentage  Imakdown 
provides  information  as  to  what  function  should  be  allocated  to  each  of  Ae  crewmembers. 
The  percentage  breakdown  directly  supptxts  the  task  workload  allocation  recommendations 
made  earlier  during  the  Function  Analysis  phase  of  this  effort  (Ward  et  al.,  1991;  Vol.  I). 
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Figure  31.  Task  percentage  comnarisons  across  all  three  missions 
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Figure  32.  Castle  task  percentage  comparisons  hv  crew  position. 
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Mission  Questionnaires 


Mission  questionnaires  (Appendices  B-D)  collected  at  the  end  of  each  mission  were  used  to 
help  determine  critical  areas  involved  with  higher  workloads  as  identified  by  the 
crewmembers.  The  questions  were  multiple  choice  with  a  section  after  each  question  for 
any  explanations/comments  an  individual  might  deem  necessary.  The  results  of  these 
questionnaires  are  summarized  below. 

Minot  Mission 

The  total  number  of  respondents  answering  the  Minot  missitm  questionnaire  was  30 
(10  pilots,  10  copilots,  and  10  navigattx^).  Over  two-thirds  of  all  respondents  felt  the 
various  mission  events  (e.g.,  late  takeoff,  communications  difficulties,  early  RZ  time) 
resulted  in  only  slight  increases  in  wtxkloads  over  a  typical  (average)  flight  mission. 
However,  the  hydraulic  malfunction  resulted  in  a  moderate  increase  in  workload.  One  pilot 
stated,  "Tlie  simulatOT's  navigation  systems  made  the  workload  (associated  with  the 
malfunction)  almost  comparable  to  what  it  would  be  in  the  airplwe  with  a  4  man  crew." 
This  indicates  that  the  increase  in  workload  resulting  from  tiie  malfunction  was  not  an 
artifact  of  the  design  but  of  the  malfunction  itself.  Other  ctxnments  suggested  that  such  a 
mission  would  have  been  easily  accomplished  with  a  similarly  design^  cockpit  Figure  33 
shows  how  each  of  the  crew  positions  responded  when  asked  if  "a  minimally  experienced 
pilot  with  a  minimally  experienced  copilot  could  have  successfully  flown  this  mission." 

The  x-axis  lists  the  crew  position;  the  y-axis  indicates  the  percentage  of  responses.  The 
respondents  (93% )  felt  confident  that  a  minimally  experienced  crew  could  have  flown  this 
mission  with  this  conceptual  cockpit  design.  This  filler  supports  the  feasibility  of  a  two* 
person  (No-Nav)  conc^tual  cockpit  design  under  lower  worUoad  conditions. 
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Fipire  33.  Minot  mission  minimally  qualified  crew  response  yranh. 
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Mather  Mission 


The  respondent  pool  included  all  of  the  subjects  (KC-10  and  KC-135  pilots,  copilots, 
and  navigators)  for  a  total  of  34  respondents.  The  subjects  reported  only  slight  increases  in 
wOTldoad  for  each  of  the  following  flight  events:  (1)  Late  takeoff,  (2)  Cell  departure  and 
join-up,  (3)  Center-directed  course  change,  (4)  Thunderstorm  avoidance,  (5)  GPS  failure, 
and  (6)  h^ssion  divert.  None  of  the  events  questioned  were  rated  as  causing  more  than  a 
slight  increase  in  workload  over  that  of  a  typical  mission.  Pilots  indicated  that  a  large 
percentage  of  the  workload  was  a  result  of  the  unfamiliarity  with  the  new  equipment  All 
of  the  pilots  felt  that  increased  training  would  have  resulted  in  lower  stress  and  workload 
levels.  Approximately  80%  of  the  crewmembers  (Figure  34)  felt  a  "minimally  qualified 
crew"  could  have  successfully  completed  this  mission.  "Missing"  refers  to  respondents 
who  failed  to  answer  the  question.  An  interesting  point  is  the  fact  that  all  of  the  KC-10 
crewmembers  felt  confident  that  they  could  have  flown  this  mission  successfully.  Keeping 
in  mind  the  similarity  of  the  training  that  KC-10  crew  members  undergo,  these  ^ta  suggest 
that  increased  training  in  equipment  use  and  navigational  techniques/procedures  would 
prove  very  effective.  These  findings  further  suggest  the  conceptual  design  is  keeping 
workload  levels  down  to  a  manageable  level. 
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FiRure  34.  Malher  mission  minimally  qualified  crew  response  graph. 
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Castle  Mission 

The  Castle  subject  pool  was  the  same  as  for  the  Mather  mission  (n=34).  Slight 
increases  in  workload  over  that  of  an  average  mission  were  reported  for  (1)  Cell  departure 
and  join-up,  (2)  Thunderstorm  avoidance,  (3)  GPS  failure,  and  (4)  Increased 
communications.  Moderate  increases  were  reported  for  (1)  Air  refueling  track  change,  (2) 
Dual  INS/GPS  failure,  and  (3)  the  Missed  approach/Weather  divert  to  Beale  AFB.  A 
substantial  increase  in  workload  was  identiti^  for  the  autopilot  failure.  This  was  attributed 
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to  the  fact  that  one  pilot  was  then  required  to  continually  fly  the  aircraft  manually.  Several 
pilots  felt  that  the  workload  associated  with  the  autopilot  f^ure  was  aggravated  as  a  result 
of  the  simulator.  Comments  like  "If  the  simulator  would  have  trimmed  up  and  remained 
trimmed  like  a  real  aircraft,  it  would  have  lessened  a  lot  of  the  extra  workload;"  and 
workload  increased  "more  so  in  the  simulator  than  in  real  life  due  to  the  lack  of  flight 
sensation;"  were  common  among  the  respondents.  As  stated  earlier  in  the  SWAT  and 
SWORD  results,  workloads  involved  with  flying  the  simulator  as  compared  to  that  of  the 
aircraft  were  always  higher.  Despite  the  increa^  workloads  associate  with  this  mission, 
76%  of  all  of  the  respondents  (Figure  35)  still  felt,  given  this  conceptual  cockpit  design,  a 
"minimally  qualified  crew"  could  have  successfully  flown  this  mission. 
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Figure  3S.  Castle  mission  minimally  qualified  crew  response  graph. 


A  review  of  Hgure  36  presents  the  overall  response  percentages  for  all  of  the 
missions  combined  by  crew  position.  In  every  instance,  KC-10  crew  members  felt  they 
could  have  successfully  completed  the  mission;  whereas,  KC-135  crew  members  were 
more  uncertain  about  such  an  endeavor.  Eighty-*five  percent  of  the  KC-135  pilots  and 
copilots  combined  felt  they  could  accomplish  the  various  missions.  KC-135  navigators 
were  less  optimistic.  Only  73%  of  the  navigatcxs  felt  a  minimally  qualified  crew  could 
have  completed  the  three  missions.  Oewmembers'  comments  explain  much  of  the 
reservations  as  to  why  a  mission  with  a  minimally  qualified  crew  might  prove 
unsuccessful;  "Extensive  training  would  be  requir^;"  "Even  with  today's  system  it's  still 
not  a  good  idea;"  and  "Experience  with  the  system.  Center,  timing,  etc.  is  what  counts;" 
These  crewmembers'  comments  indicate  their  reservations  are  not  a  result  of  the  system, 
but  more  a  byproduct  of  the  lack  of  experience.  Several  crewmembers  who  questioned  the 
likelihood  of  successful  mission  completion  also  stated  reservations  as  to  whether  a 
minimally  qualified  crew  could  have  successfuUy  flown  the  Castle  mission,  given  the 
current  aircraft  configuration  with  a  navigator.  The  overall  conclusion  based  on  the 
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missioii  specific  questionnaires  and  associated  comments  is  that  a  two-man  (No-Nav) 
oxiceptual  coclqnt  is  a  viable  approach  for  the  KC-13S. 
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FEASIBILITY  DEMONSTRATION  FINDINGS 


Results  of  the  Subjective  Workload  Assessment  Technique  (SWAT),  Subjective 
WORkload  Dominance  (SWORD)  technique,  the  Modified  Cooper-Harper  (MCH)  scale, 
the  mission  questionnaire,  and  the  objective  performance  measures  consistently  support  the 
following  findings: 

1.  The  manipulation  of  mission  difficulty  generally  resulted  in  the  expected  pattern  of 
results.  The  Minot  mission  was  easiest,  followed  by  the  Mather  mission,  and  the  Castle 
mission  was  substantially  more  difficult. 

2.  The  crewmembers'  ratings  of  the  percentage  of  time  spent  on  the  various  tasks  were 
relatively  stable  across  missions.  Additionally,  the  differences  in  percentage  of  time  spent 
on  each  task  varied  by  position  as  anticipated  by  the  Ward  et  al.  (1991)  function  analysis 
study. 

3.  The  navigator  consistently  rated  workloads  higher  than  the  pilots;  whereas,  the  KC-10 
pilots  consistendy  rated  woridoads  lower  than  their  KC-13S  counterparts. 

4.  The  vast  majority  of  crewmembers  felt  a  minimally  qualified  pilot  team  could  have 
successfully  flown  any  of  the  three  missions  encounter^,  given  the  prototype 
design/system  capabilities. 

5.  Increased  workloads  were  encountered  for  the  inflight  emergency,  random  air  refueling, 
and  weather  divert  However,  none  of  these  phases  of  flight  resulted  in  workloads  that 
were  unmanageable. 


CONCLUSION 

THE  nNDINGS  OF  THIS  STUDY  HAVE  DEMONSTRATED  THAT 
A  TWO.PERSON  (NO-NAV)  COCKPIT  IS  FEASIBLE. 

The  above  findings  and  conclusion  are  all  based  on  the  final  design  possessing 
similar  system  capabilities  and  functional  requirements  as  those  presented  in  the  Crew 
Station  Evaluation  Facility's  conceptual  cocl^it  The  extent  to  which  the  two  cockpits  (the 
CSEF  conceptual  cockpit  and  the  final  KC-135  cockpit  design)  are  similar  in  system 
capabilitiesMinctional  requirements  represents  the  extent  to  which  the  findings  are  valid.  If 
the  final  design  is  substantially  different  from  that  demonstrated  in  this  study,  the  results  of 
this  study  would  be  less  generalizable.  Based  on  this,  recommendations  for  Ae  functional 
requirements  identified  through  this  study  to  aid  system  program  engineers  in  developing 
system  specifications  are  discussed. 
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RECOMMENDATIONS 


The  following  recommendations  have  been  placed  into  three  categories  to  support 
Directorate  of  Bomb^  and  Tankers  System  Program  Office  requirements.  They  are: 

(1)  Must  haves  -  equipment/capabilities  that  are  substantial  workload  reducers  and  deemed 
afa^lutely  essential  for  mission  success.  Failure  to  include  these  capabilities  in  the  final 
cockpit  design  would  result  in  workload  levels  that  are  unmanageable  and  mission  success 
would  be  jeopardized. 

(2)  Should  haves  -  equipment/capabilities  that  are  moderate  workload  reducers  deemed 
necessary  for  mission  success.  Failure  to  include  these  capabilities  could  result  in  higher 
workload  with  the  potential  for  increased  mission  failure. 

(3)  Nice  to  Haves-  equipment/capabilities  are  workload  reducers  that  would  aid  in  mission 
success.  Failure  to  include  these  capabilities  would  not  necessarily  jeopardize  mission 
success,  but  could  increase  the  workload  levels  of  the  crewmembers. 

The  categorization  of  the  system  capabilities  were  determined  through  the  analyses  of  the 
data  presented  previously  and  from  crewmember  responses  and  comments  to  Ae  system 
questionnaire  (Appendix  E).  An  explanation  of  each  of  the  recommendations  provides  the 
rationale  for  its  inclusion  into  the  category.  Additionally,  several  issues  were  raised 
through  the  course  of  the  study  that  were  not  specifically  addressed  in  the  study.  These 
issues  are  explained  in  detail  in  the  Issues  to  Consider  section. 

Must  Haves 

1.  A  navigational  system  that  is  highly  accurate  and  reliable.  To  eliminate  the  navigator, 
we  assumed  the  system  consisted  of  two  INSs  and  a  GPS.  This  system  was  very  accurate, 
when  fully  operational.  The  loss  of  the  GPS  resulted  in  a  slight  workload  increase  for  the 
Mather  mission.  However,  the  workload  increased  to  a  moderate  workload  level  when  the 
crew  lost  both  of  the  INSs  and  the  GPS  together  (Castle  mission).  This  indicates  the  need 
for  both  an  accurate  and  reliable  navigation  system.  One  crew  stated  the  loss  of  both 
systems  caused  them  to  "consider  navigating  off  TA(}ANs  and  fly  conventionally."  This 
tacan  capability  would  probably  not  exist  in  wartime  or  for  overwater  flight  operations. 

2.  Control  Display  Units  (CDUs)  with  acceptable  sized  alphanumeric  keys,  function  select 
keys,  and  line  select  keys  to  automate  many  of  the  navigation! control  functions  previously 
held  by  the  navigator.  The  CDUs  provided  a  central  pkce  to  control  the  various  functions 
previously  perfOTtned  by  the  navigator.  Crewmembers  felt  two  CDUs  were  necessary  to 
adequately  perform  the  mission  as  this  allowed  "Each  pilot  to  have  his  own  display  and 
make  separate  inputs."  When  queried  as  to  the  size  of  the  keys,  the  pilots  felt,  "the 
oversiz^  letters/numerals  were  a  good  size  when  compared  to  the  FSAS  and  757/767. 

The  size  in  the  simulator  aids  in  seeing  the  pushbutton  keys  in  dark  conditions." 
Consequently,  the  size  reduced  the  time  ne^ed  in  key  recognition/key  stroke.  The  other 
features  of  the  CDU  were  all  deemed  as  either  moderately  or  completely  acceptable 
indicating  the  CDUs  were  interfaced  adequately. 
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3.  A  ground  Mission  Planning  System  (MPS)  with  the  capability  to  formulate  the 
necessary  database  for  a  given  mission  during  mission  planning  day  and  then  be 
tran^erable  into  the  aircraft  via  a  floppy  disk  or  similar  type  system.  An  assumption  made 
£rom  the  start  of  the  study  was  that  an  MPS  would  be  simulated  that  could  load  the  entire 
database  into  the  Mission  Management  System  for  each  flight.  If  a  crew  was  required  to 
manually  load  the  datapoints  for  each  flight,  an  increase  in  time,  workload,  and  human 
enor  would  result  Additionally,  should  an  inflight  malfunction  occur,  the  crew  might  be 
forced  with  manually  reloading  the  entire  database.  An  MPS  with  the  above  capability 
would  greatly  reduce  this  associated  workload. 

4.  An  intercom  system  that  allows  the  individual  to  tune  the  volume  of  each  radio  to  the 
level  desired,  as  radios  will  be  controlled  through  the  CDUIRRU  (Remote  Readout  Unit). 
Critical  to  the  success  of  the  communication  system  designed  for  the  conceptual  cockpit 
was  the  need  to  replace  the  older,  less  capable  interphone  system.  Since  the  radio 
powerheads  were  removed  from  the  overhead  panel  and  all  radio  power  was  controlled 
through  the  CDU,  radio  volume  control  ne^^  to  be  controlled.  Previous  flight  crews 
had  expressed  several  complaints  about  the  inability  to  individually  tune  the  l^IF  radios. 
Given  a  design  change,  and  the  availability  of  off-Ae-shelf  intercom  systems,  the  design 
incorporated  an  int^hone  system  that  allowed  fm*  each  crewmember  to  individually  tune 
their  radios.  The  pilots  found  this  type  of  interphone  system  to  be  essential  to  mission 
success. 

5.  A  Mission  Management  System  with: 

a.  Easy  to  understand  symbol  conventions . 

b.  Lefi/Right  scrolling  between  display  pages  and  UplDown  scrolling  between 
pages.  A  pilot  study  performed  prior  to  the  actual  study  indicated  the  actual 
scrolling  of  display  pages  needed  to  be  consistent  within  the  MMS.  The  pilot  study 
indicate  that  leftMght  scrolling  was  most  suitable  for  scrolling  between  pages. 
Scrolling  between  pages  refers  to  scrolling  between  pages  that  are  function  Jly 
different  (e.g..  Power  page  to  Start  pages  to  Take  off  and  Landing  pages,  etc.). 
Up/Down  scrolling  was  assigned  to  scrolling  within  functional  pages  (e.g..  Start  1 
page  to  the  Start  2  page,  etc.). 

c.  Waypoint  database  large  enough  to  fly  long  combat  type  missions.  The  database 
should  be  large  enough  to  allow  for  the  complete  mission  route  of  flight  Several 
comments  identified  the  inadequacy  of  the  current  KC-135  inertial  navigation 
system  as  being  "too  limiting"  and  "cumbersome."  A  larger  database  would  allow 
for  increased  information  to  be  inserted  during  mission  planning  day  under  low 
stress  conditions  resulting  in  lower  inflight  workloads. 

d.  Mission  database  that  includes  2-,  3-,  4-,  and  5 -letter  ICAO  identifiers,  tacans, 
airfields,  intersections;  air  refueling  tracks;  and  Military  Operating! Restricted  areas 
for  increased  situation  awareness,  reduction  in  crew  workload,  and  ease  of  irflight 
replanning.  The  ability  to  address  action  points/geographical  locations  by  their 
associated  ICAO  identifiers,  MOA  names,  and  track  numbers,  instead  of  strictly 
by  latitude  and  longitude  was  identified  by  the  crewmembers  as  a  major  workload 
saving  capability  used  during  inflight  planning. 

e.  Communication,  Nov  Aid,  IFF  Navigational  Status,  Direct  To,  Flightplan, 
Update  Rendezvous! Orbit,  Airborne  Directed  Approach,  and  Fuel  Management 
capability  pages,  each  tied  to  a  dedicated  function  select  key.  The  results  from  both 
a  previous  pilot  study  and  the  present  study  indicated  that  head  down  time  was 
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related  to  which  functions  were  assigned  dedicated  keys.  Several  off-the-shelf 
systems  have  the  orbit  function  tied  to  a  lower  level  menu  and  not  directly  to  a 
function  select  key.  Given  the  KC-135  mission,  the  conceptual  cockpit  desi^ 
assigned  the  rendezvous/orbit  function  its  own  dedicated  key.  This  resulted  in 
decreased  head  down  time  and  lower  workload.  Similar  results  were  found  for 
the  other  functions  listed  above. 

f.  RendezvouslOrbit  capability  providing  the  pilot  with  a  pictorial  view  of  his  route 
offlighttorbit  with  associated  turn  range  depicted,  independent  of  the  air-to-air 
tacan.  Turn  range  and  offset  should  be  computer  generated  to  further  reduce  crew 
workload  during  air  refueling,  a  heavy  workload  time  period.  Crewmembers 
found  this  capability  as  essential  to  keep  workloads  at  a  manageable  level.  One 
crew  member  stated  "This  function  was  necessary  without  a  navigator."  Another 
pilot  claimed  it  was,  "A  great  idea  to  visually  display  the  orbit  area."  The  pilots 
identified  the  manud  requirement  to  update  the  offset  and  turn  range  when  it 
differed  from  what  was  originally  set,  as  a  limiting  factor  of  this  design.  One  pilot 
stated,  "The  system  counts  too  much  on  pilot  input"  for  the  rendezvous/orbit 
function,  while  another  pilot  felt  all  data  relevant  to  calculating  turn  range  and  offset 
should  be  automated,  "No  (unnecessary)  thinking  should  be  done  by  the  pilots." 

A  KC-10  pilot  stated,  "The  KC-10  system  figures  out  turn  range  and  offset ,  all  we 
have  to  teU  it  is  receiver  true  airspeed"  suggesting  air  refueling  workload  would  be 
greatly  decreased  if  turn  range/offset  were  automated. 

g.  A  mode  for  receiver  identification  is  required  and  should  be  controlled  through 
the  CDU.  When  queried  as  to  the  need  for  the  APN-69,  approximately  two-thirds 
of  the  respondents  felt  the  need  for  the  APN-69  beacon  still  existed.  Several  pilots 
stated  the  need  for  the  APN-69  beacon  was  due  to  long  range  cell  activity  and  for 
the  air  refueling  rendezvous.  In  contrast,  only  four  pilots  felt  they  could 
successfully  accomplish  an  air  refueling  rendezvous  with  the  air-to-air  tacan  only. 
They  genei^y  felt  the  air-to-air  tacan  without  an  azimuth  capability  was  limited  in 
its  functional  value.  Consequently,  some  additional  means  of  receiver  identification 
besides  the  air-to-air  tacan  would  be  beneficial.  In  summary,  the  need  for  a  mode 
of  receiver  identification  exists  and  should  be  controlled  through  the  CDU,  as  all 
other  air  refueling  functions  are  controlled  through  the  CDU. 

6.  A  color  weather  radar  display  with: 

a.  Separate  radar  range  controls  for  each  of  the  pilots.  Eighty-seven  percent  of  the 
pilots  agreed  separate  controls  for  radar  range  should  be  incorporated.  They  cited 
the  ne^  to  conduct  cell  and  rendezvous  operations  close-in,  while  simultaneously 
scanning  weather  at  a  distance,  as  a  major  limitation  of  the  CSEF  conceptual  coc^it 
design,  which  only  had  one  set  of  radar  controls.  KC-10  pilots,  who  currently  fly 
with  two  separate  radar  controls,  stated  emphatically  that  two  separate  control 
panels  would  be  "most  definitely"  needed.  Further  clarification  of  the  answers 
indicated  that  a  single  control  panel  often  led  to  confusion  and  increased  workload 
levels. 

b.  A  radar  display  on  the  Electronic  Horizontal  Situational  Indicator. 

Crewmembere  did  not  believe  the  need  for  a  dedicated  radar  was  necessary.  Again, 
87%  of  the  respondents  preferred  the  radar  display  on  the  EHSI.  One  pilot 
summed  up  the  lack  of  a  need  for  a  dedicated  radar  display,  "It  is  good  displaying  it 
(radar  presentation)  on  the  EHSI  since  that’s  what  the  pilot  looks  at."  However,  in 
order  to  successfully  perform  radar  iterations  at  a  reduced  workload  level,  pilots 
strongly  indicated  that  a  color  weather  radar  display  would  be  essential.  They  did 
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not  believe  radar  operations  would  be  manageable  if  a  radar  display  siinilar  to  that  in 
the  KC-13S  aircraft  were  used.  Comments  such  as,  "Color  WX  radar  definitely 
decreases  woridoad  and  mission  difficulty,"  and  "Absolutely  necessary"  were 
indicative  of  the  majority  of  the  responses  regarding  the  need  fen*  a  color  radar. 
Additionally,  the  KC-10  pilots  who  all  had  previous  experience  with  both  a  color 
and  a  monochrome  radar  display  stated,  "Color  radar  is  definitely  the  best,"  further 
supporting  the  need  for  a  color  radar  display  on  the  EHSI. 

c.  The  receiver  identification  mode  must  be  fully  integrated  via  the  CDU  and  the 
radar  presentation  on  the  EHSI.  The  importance  of  receiver  identification  mode 
displayed  on  the  EHSI  was  explained  in  5g  above. 

7.  An  EHSI  with  the  following  capabilities  incorporated  at  both  the  pilot  and  copilot 

stations. 


a.  Full  compass  (360  degrees)  and  arc  (approximately  90  degrees)  map  display 
capability.  The  system  was  evaluated  with  both  capabilities.  Each  capability  was 
deemed  essential  for  mission  success.  The  study  results  indicated  the  pilot  flying 
the  aircraft  generally  had  the  full  compass  displayed,  except  for  weather  avoidance 
and  cell  departure  and  join-up.  The  pilot  not  flying  the  aiitnaft  typicaUy  had  an  arc 
display  presented  to  increase  his  navigational  ability  and  situational  awareness. 

The  pUots  stated  this  capability  gave  ^em  increased  situational  awareness  and 
reduced  overall  workload. 

b.  Ability  to  overlay  waypoints,  navaids,  and  airports.  Pilots  rated  the  ability  to 
overlay  waypoints,  navaids,  and  airports  directly  onto  the  EHSI  as  a  substantial 
overall  workload  reducer.  When  sj^flcally  asked  which  design  functions  were 
the  most  beneficial  in  accomplishing  the  mission,  this  capability  was  listed  noost 
often.  The  pilot  stated  it  allowed  them  the  oppmtunity  to  literally  "see  the  big 
picture"  resulting  in  increased  situational  awareness  and  decrea^  woddoad. 

c.  Full-time  course,  distance,  crosstrack,  cmd  time  to  go  (TTG)  readouts.  A 
full-time  readout  of  the  functions  listed  was  deemed  necessary  to  keep  workloads 
down.  The  CSEF  design  provided  the  informatio;  however,  the  information  was 
not  presented  full-time.  It  was  presented  on  a  rotary  switch  with  four  other  pieces 
of  formation  or  on  different  CDU  display  pages  (e.g.,  INAV  page).  The  pilots 
felt  the  constant  switching  to  obtain  this  information  was  both  time-consuming  and 
detrimental  to  their  situational  awareness,  since  it  required  them  to  locate  the  rotary 
switeh  on  the  overhead  panel.  They  felt  a  full-time  presentation  of  the  above 
information  would  provide  them  with  the  information  most  often  provided  to  them 
by  the  navigator. 

d.  Capability  to  display  Groundspeed  (GS),  True  Airspeed,  Wind  Direction  and 
Velocity  (WIV),  and  Drift  Angle.  The  nwd  for  the  information  listed  above  was 
identified  for  various  phases  of  flight.  The  true  airspeed  enabled  the  pilots  to 
better  maintain  airspeed  at  altitude.  Groundspeed  was  effectively  used  during 
approach  and  landing  and  for  estimated  titiK  of  arrival  calculations  used  to  backup 
the  navigation  system.  Wind  information  was  important  for  alter  heading 
calculations  and  pilot  reports  of  the  weather.  Each  piece  of  information  was 
considered  very  valuable  to  the  pilots  and  resulted  in  perceived  lower  workloads. 

The  irrqrortance  of  the  EHSI  capabilities  and  associated  information  is  best  surruned 
up  in  the  following  statements  by  a  KC-10  pilot  "The  combination  of  all  the  information 
on  the  HSI  is  great!  When  does  the  KC-10  get  this  EHSI?" 
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Should  Haves 


1.  A  Remote  Readout  Unit  (RRU)  with  rapid  radio  and  navigation  aid  frequency  change 
capability.  The  unit  should  have  its  radio  select  buttons  in  an  easily  accessible  location. 
Additiorudly,  the  pilots  should  be  able  to  tune  the  navaids  with  either  the  frequency  or  the 
ICAO  iden^fier.  The  unit  should  also  have  the  capability  to  recall  the  last  tuned  frequency 
rapidly.  Pilots  stated  the  RRU  and  its  capabilities  were  "Very  handy  in  the  integration  of 
pilot/ct^ilot  duties."  They  also  claimed  the  RRU  allowed  for  increased  crew  flexibility. 
The  cai»bility  to  recall  the  last  tuned  frequency  at  die  press  of  the  button  was  recogniz^  as 
a  teal  time-saver.  They  did,  however,  suggest  the  unit  be  reduced  a  little  in  size  a^ 
concerns  over  whether  the  current  color  ted  would  be  suitable  for  all  conditions  of  flight, 
specifically,  bright  sunlight  were  expressed  (See  Issues  to  Consider). 

2.  An  Electronic  Attitude  Director  Indicator  (EADI)  that  resembles  pictorially  the  FD-I09 

system  currently  in  the  plane.  It  should  also  have  a  radar  altitude  decision  height 

readout  capability.  The  ability  to  display  digitally  the  radar  altitude  and  decision  height  was 
seen  as  very  beneficial  by  the  pilots.  The  pilots  generally  felt  this  helped  their  perframance 
by  reducing  the  rieed  to  check  the  analog  r^ar  altimeter.  They  also  felt  the  presentation  of 
the  letters  "DH"  in  the  left  center  of  the  EADI  provided  them  valuable  information  at  a 
critical  time  in  flight,  although  one  pilot  suggested  a  flashing  'DH"  of  a  different  color 
nught  be  more  of  an  attention-getter.  AnoAer  issue  raised  by  one  pilot,  but  beyond  the 
scope  of  this  study,  was  that  of  the  increased  reliability  and  maintainability  of  the  EADI 
over  the  old  analog  ADI. 

3.  A  Fuel  Savings  Advisory/Caution  Advisory  System  fully  integrated  through  the  CDU 
should  be  explored  because  such  a  system  could  greatly  reduce  their  workload  over  the 
current  system.  Due  to  the  direction  given  the  CSBF  for  the  development  of  the  conceptual 
design,  the  Fuel  Savings  Advisory  System  was  not  integrated  through  the  CDU.  This  led 
to  increased  workload  among  the  pilots  because  pilots  were  often  forced  to  perfram  manual 
calculations  due  to  the  lack  of  system  integration.  Additionally,  not  integrating  the  system 
through  the  CDUs,  would  result  in  an  additional  CDU  to  operate  the  FSA/CAS,  costing 
mote  nxrney  and  taking  up  valuable  cockpit  space. 

Ni«  iQ  Haves 

1.  A  master  caution  light  to  enhance  malfunction  recognition  in  accordance  with  standard 
miliutry  practice.  A  warningl caution  advisory  panel  although  not  a  necessity,  should  be 
considered  as  the  pilots  would  then  have  only  one  central  place  to  look  in  case  of  a  master 
caution  light  illumnating,  decreasing  recognition  time.  Approximately  90%  of  the 
respondents  cemsidered  the  Master  Caution  light  as  very  effective.  The  respondents  also 
believed  the  analysis  of  the  malfunction  was  sped  up  by  immediate  recognition  of  the 
problem  via  the  waming/caution  advisory  panel.  However,  the  pilots  did  feel  the  current 
system  was  adequate  for  most  malfunctions  and  resulted  in  only  slightly  higher  workloads. 

2.  A  digital  countdown  timer  would  aid  the  pilots  during  various  phases  of  flight  (i.e.,  non¬ 
precision  of^roach).  This  function  was  considered  as  nice  to  have  because  it  enabled 
pilots  to  make  accurate  timings  for  missed  approach  procedures. 

3.  Angle  of  Attack,  Indicated  Airspeed,  Barometric  Pressure,  and  Vertical  Velocity 
displayed  in  the  upper  corners  of  the  EADI  would  help  reduce  workload  by  lowering  cross 
check  scan  range.  This  information  was  liked  by  the  pilots  because  it  enabled  them  to 
focus  nx>re  on  the  ADI,  a  primary  flight  reference.  However,  the  majority  of  the  pilots  did 
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not  feel  it  to  be  a  necessity  for  mission  success,  since  it  was  mainly  redundant  informadon 
that  could  be  (Stained  from  the  analog  instniments. 

Issues  to  Consider 

Throughout  the  study,  several  issues  woe  raised  that  could  not  be  resolved  fn»n 
the  results  of  Ae  study.  These  issues  were  either  not  addressed  by  the  study  or  insufficient 
data  were  collected  to  draw  substantive  ccmclusions.  The  following  is  a  list  of  those  issues: 

1.  Mostsuitablerangesandscalesfor  the  weather  radar.  The  particular  ranges  and  scales 
for  each  range  selection  must  be  addressed.  The  baseline  system  provided  various  ranges 
with  mid-range  scales/markers  (e.g.,  the  S-mile  ran^  had  a  mid-range  scaleAnaker  of  2.S). 
An  earlier  pilot  study  indicated  that  the  5-  mile  range  should  have  l-i^e  scale  markers  to 
assist  in  cell  departure  and  join-up  and  rendezvous  procedures.  Similar  determinations  of 
the  other  ranges  and  scales  are  necessary  to  develop  the  best  radar  presentations. 

2.  Ambient  lighting  considerations  for  the  EADls,  EHSIs,  CDUs,  and  RRUs.  The  study 
was  conducted  under  simulated  nighttime  environments.  No  evaluation  was  made 
ctxiceming  potential  washout  effects  of  sunli^t  or  color  degradation  due  to  night 
goggles/hel^t  visors.  Future  studies  should  address  these  potential  problems. 

3.  Increased  ground  training  requirements  for  each  pilot  in  navigation,  radar  procedures, 
mission  planning,  and  inflight  replanning.  The  most  common  comment  by  pUots 
throughout  the  course  of  the  study  concerned  the  limited  amount  of  training.  They  felt  a 
large  percentage  of  tiie  woikload  they  experienced  was  the  result  of  minimi  training  time. 
AMtionally,  given  the  navigator  is  trained  ftxr  an  extensive  period  of  time  (as  much  as  18 
months  in  some  aircraft)  and  the  pilots  will  be  required  to  pmmn  what  were  previously 
navigator  duties,  it  is  reasonable  to  assume  incret^ed  training  will  be  required  and  should, 
therefore,  be  considered. 

4.  A  full  radar  presentation  for  the  EHSI  map  display  may  increase  situational  awareness. 
If  both  pilots  are  displaying  a  full  compass  rose,  ^n  the  potential  for  inadvertent 
thunderstonn  penetration  exists  because  the  conceptual  design  did  not  allow  radar  mapping 
to  be  displayed  in  full  compass  rose.  This  is  a  potential  problem  in  the  transition,  ^)proach 
aiui  landing  phases  of  flight  when  both  pilots  typically  had  the  full  compass  rose  di^layed. 

5.  An  altitude  warning  signal.  Given  the  potential  for  altitude  deviations  and  the  removal 
of  the  additional  set  of  eyes  in  the  cockpit  (i.e.,  navigator),  consideration  should  be  given 
fOT  the  possible  inclusion  of  an  altitude  warning  sign^.  Although  this  may  be  an  artifact  of 
inadequate  training,  an  evaluation  for  the  need  of  such  a  signal  should  be  performed. 
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SUMMARY 


This  study  was  part  three  of  a  three-phase  effort  to  demonstrate  the  feasibility  of  a 
two-person  (No-nav)  conceptual  coclq)it  design.  Ten  KC-13S  and  2  KC-10  crews  flew 
seve^  different  missions  over  a  1-week  period.  Over  the  course  of  the  week,  various 
subjective  (SWAT,  SWORD,  and  MC!H)  and  objective  measures  (e.g.,  airspe^,  aldtutfe) 
were  used  to  demonstrate  that  such  a  design  was  feasible.  Additionally,  the  subjective  and 
objective  measures  were  used  in  conjunction  with  questionnaire  data  to  derive  a  set  of 
functicxial  requirements.  These  requirements  were  placed  into  one  of  three  categories: 

Must  haves.  Should  haves,  and  Nice  to  haves.  A  recap  of  each  is  shown  in  Table  6. 

From  the  st^y  also  sprung  several  issues  of  concern.  Hiese  issues  were  not  specifically 
addressed  within  the  realm  of  the  snidy,  and  should  be  considered  in  any  future 
developmental  efforts. 


Table  6.  Functional  requirements  and  their  categorical  assignment. 

_ MUST  HAVE _ SHOULD  HAVE _ NICE  TO  HAVE 

WARNING  ADVISORY  PANEL 

DIGITAL  COUNTDOWN  TIMER 

EADI  DKiTTAL  READOUTS 

NAVWATIWr  SYSTEM 
INTERPHONESYSTIEM 
COLOR  WX  RADAR 
EHSb 


MISSION  MANACXMENT  SYSTEM  REMOTE  READOUT  UNIT 

C(»frROLE«SPLAYUNrrS  EADIs 

MISSION  PLANNING  SYSTEM  INTEGRATED  FSA/CAS 
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APPENDIX  A 
SWORD  RATING  FORMS 
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MINOT  Mission  (1)  -  Cruise  Segment  (1)  ^ 
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MATHER  Mission  (2)  -  Weather/Divert  (6) 
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APPENDIX  B 


MINOT  MISSION  QUESTIONNAIRE 
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Minot  Mission 


Questionnaire 


This  quesdcmnaiie  is  a  mission  specific  quesdcxinaiie  (xxiceming  the  various  events  and 
actions  undergone  during  the  last  mission.  You  should  answer  the  questionnaire  from  your 
own  individu^  perspe^ve  by  circling  the  impropriate  answer.  If  you  feel  that  any  questicm 
needs  furtiier  e^lanation,  please  feel  free  to  ask  (me  of  the  experimnters  for  clarification.  If 
you  feel  no  one  answer  is  ^equate,  please  use  the  comments  secticm  afrer  each  questicm  to 
elaborate  on  it  A  ccxnments  section  has  been  provided  after  each  question  to  aUow  you  to 
actively  express  all  concerns  you  might  have  about  a  ^ven  question,  mission,  or  instrument 
You  are  encouraged  to  use  the  comments  section  whenever  possible.  However,  do  not  feel 
you  must  comment  on  every  question.  For  those  questions  requiring  more  space  than  that 
provided,  simply  turn  die  page  over  and  write  on  the  back.  Additio^  comment  space  is  also 
provided  on  the  last  page  of  the  questionnaire. 
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PERSONAL  DATA 

Name  (Optional): _ 

Grade:  0-1  0-2  0-3  0-4  0-5  0-6 

Age: _  Sex: _ 

Aeronautical  Rating:  Pilot  Senior  Pilot  Master  Pilot 

Nav  Senior  Nav  Master  Nav 

Crew  Position:  Nav  IN  CP  P/AC  IP 

Organization: _ 

Duty  Station: _ 

Total  Flying  Hours:  _ 

Total  KC-135  Flying  Hours:_ _ 

Total  Hours  Current  Crew  Position: _ 

Time  Since  Last  Flight:  Months  Days 

KC-13S  Aircraft  Model  Currently  Flying: _ 
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Minot  Mission 

1.  The  late  takeoff  caused  (a) _ increase  in  mission 

difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Motkrate  d.  Substantial 

Comments; _ 


2 .  Were  you  able  to  identify  your  receiver  on  radar  prior  to  your  turn  inbound 
to  the  ARIP? 

a.  Yes  b.  No 

Comments: _ 


3.  The  receiver's  early  arrival  at  the  ARIP  caused  (a) _ increase  in 

mission  difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


4.  The  communication  difficulties  encountered  at  EAR  caused  (a) 
increase  in  mission  difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 
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5.  The  hydraulic  failure  caused  (a) _ increase  in  mission 

difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Gnmnents: _ 


6 .  What  type  of  work-around  procedures  were  used  to  overcome  the 
difnculties  encountered  during  this  mission? 

Comments: _ 


7 .  Did  you  encounter  any  other  problem  areas  during  this  mission? 
(Please  explain  in  comments  section) 

a.  Yes  b.  No 

Comments: _ 


8.  Which  pieces  of  equipment  and/or  design  functions  were  particularly 
helpful  in  accomplishing  this  mission?  (Please  explain  in  comment  sections) 

Comments: _ 
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9.  Which  pieces  of  equipment  and/or  design  functions  were  extremely  hard  to 
use  and,  consequently,  caused  high  workload?  (Please  explain  in  comments 
section) 

Comments: _ 


10.  Please  recommend  any  improvements  to  the  current  equipment 

design/interface  that  you  feel  would  improve  aircrew  efficiency  and  reduce 
aircrew  workload? 

Comments: _ 


11.  What  adjective  best  describes  the  overall  difficulty  of  this  mission? 

a.  Easy  b.  Medium  c.  Hard 

12.  Using  the  modified  Cooper-Harper  (C-H)  scale  provided,  how  would  you 
rate  the  mission  just  flown? 

123456789  10 

13.  What  percentage  of  your  total  workload  came  from  each  of  the  following 
tasks?  Using  the  modified  C-H  scale,  rate  your  workload  for  each  task. 


%  Rating 

Aerial  Refueling  Tasks  _  _ 

Communication  Tasks  _  _ 

Navigation  Tasks  _  _ 

Paperwork  Tasks  _  _ 

Piloting  Tasks  _  _ 

Other  _  _ 
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14.  For  tbe  previous  mission,  rate  your  workload  as  compared  to  what  you 
think  it  would  have  been  with  the  present  KC>135  system  and  a  navigator. 
With  the  system  that  I  just  flew  my  workload  was _ . 

a.  Substantially  decreased 

b.  Moderately  decreased 

c.  Slightly  decreased 

d.  Not  changed 

e.  Slightly  increased 

f.  Modcr^ly  increased 

g.  Substanti^y  increased 

Comments: _ 


15.  Provided  adequate  training,  could  a  minimally  experienced  pilot  with  a 
minimally  experienced  copilot  successfully  fly  this  mission? 

a.  Yes  b.  No 

Comments: _ 


16.  Given  the  mission  just  flown,  could  a  single  pilot  (i.e.,  one  pilot  is 
incapacitated)  have  performed  thb  mission? 

a.  Yes  b.  No 

Comments: _ 
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17.  The  following  space  is  provided  for  you  to  elaborate  on  questions  1-16 
w  for  you  to  identify  any  other  concerns  that  you  might  have. 


APPENDIX  C 

MATHER  MISSION  QUESTIONNAIRE 
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Mather  Mission 
Questionnaire 


This  questionnaire  is  a  mission  specific  questionnaire  concerning  the  various  events  and 
actions  undergone  during  the  last  mission.  You  should  answer  the  questionnaire  from  your 
own  individui  perspective  by  circling  the  appropriate  answer.  If  you  feel  that  any  question 
needs  further  explanation,  please  feel  free  to  ask  one  of  the  experimenters  for  clarification. 
If  you  feel  no  one  answer  is  adequate,  please  use  the  comments  section  after  each  questicxi 
to  elalmrate  on  it  A  comments  section  has  been  provided  after  each  question  to  allow  you 
to  actively  express  all  concerns  you  might  have  about  a  given  question,  mission,  or 
instrument  You  are  encouraged  to  use  the  comments  section  whenever  possible. 

However,  do  not  feel  you  must  comment  on  evay  question.  For  those  questions  requiring 
more  space  than  that  provided,  simply  turn  the  page  over  and  write  on  the  back.  Additional 
comment  space  is  also  provide  on  the  last  page  of  the  questionnaire. 
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PERSONAL  DATA 


Name  (Optional):. 


Grade:  0-1  0-2  0-3  0-4  0-5  0-6 


Age: 


Sex: 


Aeronautical  Rating:  Pilot  Senior  Pilot  Master  Pilot 

Nav  Senior  Nav  Master  Nav 


Crew  Position:  Nav  IN  CP  P/AC  IP 

Organization: _ 

Duty  Station: _ 

Total  Flying  Hours: _ 

Total  KC-135  Flying  Hours: _ 

Total  Hours  Current  Crew  Position: _ 

Time  Since  Last  Flight: _  Months  _ Days 

KC-135  Aircraft  Model  Currently  Flying: _ 
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Mather  Mission 


1.  The  late  takeoff  caused  (a) _ increase  in  mission  difficulty/crew 

workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


2.  The  cell  departure/join-up  requirement  caused  (a) _ increase  in 

mission  difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


3.  The  implementation  of  the  center  directed  course  change  caused  (a) 
_  increase  in  mission  difflculty/crew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments:  _ 


4.  Thunderstorm  avoidance  caused  (a) _ increase  in  mission 

difficuity/crew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 


Comments: 


5 .  Prior  to  GPS  failure,  did  you  detect  your  INSs  were  drifting?  (Please 
explain  when  and  how  you  detected  your  INSs  were  drifting  in  the  Comments  section). 

a.  Yes  b.  No 

Comments: _ 


6.  The  failure  of  the  GPS  system  caused  (a) _ increase  in  aircrew 

workload? 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


7.  The  Command  Post  imposed  divert  to  Castle  AFB  caused  (a) 
increase  in  mission  difficulty/aircrew  workload? 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


8.  Which  pieces  of  equipment  and/or  design  functions  were  particularly 
helpful  in  accomplishing  this  mission?  (Please  explain  in  comment  sections) 

Comments: _ 
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9.  Which  pieces  of  equipment  and/or  design  functions  were  extremely  hard  to 
use  and,  consequently,  caused  a  high  workload?  (Please  explain  in  comments 
section) 

Conunents: _ 


10.  Please  recommend  any  improvements  to  the  current  equipment 

design/interface  that  you  feel  would  improve  aircrew  efficiency  and  reduce 
aircrew  workload? 

Comments: _ 


11.  What  adjective  best  describes  the  overall  difficulty  of  this  mission? 
a.  Easy  b.  Medium  c.  Haid 


12.  Using  the  modified  Cooper-Harper  scale  provided,  how  would  you  rate 
the  mission  just  flown? 

123456789  10 


13.  What  percentage  of  your  total  workload  came  from  each  of  the  following 
tasks.  Using  the  modified  C-H  scale,  rate  your  workload  for  each  task? 


%  Rating 

Aerial  Refueling  Tasks  _  _ 

Communication  Tasks  _  _ 

Navigation  Tasks  _  _ 

Paperwork  Tasks  _  _ 

Piloting  Tasks  _  _ 

Other  _  _ 


14.  For  the  previous  mission,  rate  your  workload  as  compared  to  what  you 
think  it  would  have  been  with  the  present  KC-13S  system  and  a  navigator. 
With  the  system  that  I  just  flew  my  workload  was _ 

a.  Substantially  decreased 

b.  Moderately  decreased 

c.  Slightly  decreased 

d.  Not  changed 

e.  Slightly  increased 

f.  Moderately  increased 

g.  Substantially  increased 

Conunents:  _ _ 


15.  Provided  adequate  training,  could  a  minimally  experienced  pilot  with  a 
minimally  experienced  copilot  successfully  fly  this  mission?  .« 

a.  Yes  b.  No 


16.  Given  the  mission  just  flown,  could  a  single  pilot  (i.e.,  one  pilot  is 
incapacitated)  have  performed  this  mission? 

a.  Yes  b.  No 

Comments: _ _ _ _ _ 
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17.  The  following  space  is  provided  for  you  to  elaborate  on  questions  1-16 
or  for  you  to  identify  any  other  concerns  that  you  might  have. 


APPENDIX  D 

CASTLE  MISSION  QUESTIONNAIRE 
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Castle  Mission 


Questionnaire 


This  questionnaire  is  a  mission  specific  questicmaire  ccmceming  the  various  events  and 
actions  undergone  during  the  last  mission.  You  should  answer  the  questionnaire  from  your 
own  individu^  perspective  by  circling  the  appropriate  answer.  If  you  feel  that  any  questicxi 
needs  further  ej^laiiation,  please  feel  free  to  ask  one  of  the  experimenters  for  cla^cation.  If 
you  feel  no  one  answer  is  adequate,  please  use  the  comnoents  section  after  each  question  to 
elatorate  on  it  A  comments  section  has  been  provided  after  each  question  to  allow  you  to 
actively  express  all  concerns  you  might  have  about  a  given  question,  mission,  or  instrument 
You  are  encouraged  to  use  the  comments  section  whenever  possible.  However,  do  not  feel 
you  must  comment  on  every  question.  For  those  questions  requiring  more  space  than  that 
provided,  simply  turn  the  page  over  and  write  on  the  back.  Additinnal  ccxnment  space  is  also 
provided  on  the  last  page  of  the  questionnaire. 
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PERSONAL  DATA 


Name  (Optional):. 


Grade:  0-1  0-2  0-3  0-4  0-5  0-6 


Age:. 


Sex: 


Aeronautical  Rating:  Pilot  Senior  Pilot  Master  Pilot 

Nav  Senior  Nav  Master  Nav 


Crew  Position:  Nav  IN  CP  P/AC  IP 

Organization: _ 

Duty  Station: _ 

Total  Flying  Hours: _ 

Total  KC-135  Flying  Hours: _ 

Total  Hours  Current  Crew  Position: _ 

Time  Since  Last  Flight: _  Months  _ Days 

KC-135  Aircraft  Model  Currently  Flying: _ 
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Castle  Mission 


1.  The  autopilot  failure  caused  (a) _ increase  in  mission 

difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


2.  The  cell  departure/join>up  requirement  caused  (a) _ increase  in 

mission  difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


3 .  The  air  refueling  area  change  caused  (a) _ increase  in  mission 

difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


4.  Thunderstorm  avoidance  caused  (a) _ increase  in  mission 

difficulty/crew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 
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5 .  Prior  to  GPS  failure,  did  you  detect  your  INSs  were  drifting?  (Please 
explain  when  and  how  you  detected  your  INSs  woe  drifting  in  the  Comments  section). 

a.  Yes  b.  No 

Comments: _ 


6.  The  failure  of  the  GPS  system  caused  (a) _ increase  in  mission 

difficulty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ : _ 


7.  The  failure  of  both  INS  systems  caused  (a) _ increase  in  mission 

difnculty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


8.  The  weather  divert  to  Beale  AFB  caused  (a) _ increase  in  mission 

difficuity/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 


(Comments: 


9 .  The  ability  to  call  up  individual  air  refueling  tracks/military  operating 
areas  caused  (a) _  decrease  in  aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Gjniments: _ 


10.  The  ability  to  fly  to  an  airport/IAF  by  the  insertion  of  the  airport/IAF 

identifier  during  a  weather  divert  caused  (a) _ decrease  in  aircrew 

workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments: _ 


11.  The  increased  volume  of  radio  traffic  and  radio  frequency  changes  caused 
(a)  _  increase  in  mission  di^culty/aircrew  workload. 

a.  No  b.  Slight  c.  Moderate  d.  Substantial 

Comments; _ 


12.  Which  pieces  of  equipment  and/or  design  functions  were  particularly 
helpful  in  accomplishing  this  mission?  (Please  explain  in  comment  sections) 

Comments; _ 
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13.  Which  pieces  of  equipment  and/or  design  functions  were  extremely  hard  to 
use  and,  consequently,  caused  a  high  workload?  (Please  explain  in  comments 
section) 

Comments: _ 


14.  Please  recommend  any  improvements  to  the  current  equipment 

design/interface  that  you  feel  would  improve  aircrew  efficiency  and  reduce 
aircrew  workload? 

Comments: _ 


15.  What  adjective  best  describes  the  overall  difficulty  of  this  mission? 
1.  Easy  b.  Medium  c.  Hard 


16.  Using  the  modified  Cooper-Harper  (C-H)  scale  provided,  how  would  you 
rate  the  mission  just  flown? 

123456789  10 


17.  What  percentage  of  your  total  workload  came  from  each  of  the  following 
tasks.  Using  the  modifled  C-H  scale,  rate  your  workload  for  each  task? 

%  Rating 

Aerial  Refueling  Tasks  _  _ 

Communication  Tasks  _  _ 

Navigation  Tasks  _  _ 

Paperwork  Tasks  _  _ 

Piloting  Tasks  _  _ 

Other  _  _ 
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18.  For  the  previous  mission,  rsite  your  workload  as  compared  to  what  you 
think  it  would  have  been  with  the  present  KC-13S  system  and  a  navigator. 
With  the  system  that  I  just  flew  my  workload  was _ 

a.  Substantially  decreased 

b.  Moderately  decreased 

c.  Slightly  decreased 

d.  Not  changed 

e.  Slightly  increased 

f.  Moderately  increased 

g.  Substanti^ly  increased 

Comments: _ 


19.  Provided  adequate  training,  could  a  minimally  experienced  pilot  with  a 
minimally  experienced  copilot  successfully  fly  this  mission? 

a.  Yes  b.  No 

Comments: _ 


20.  Given  the  mission  just  flown,  could  a  single  pilot  (i.e.,  one  pilot  is 
incapacitated)  have  performed  this  mission? 

a.  Yes  b.  No 

Comments: _ 
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21.  The  following  space  is  provided  for  you  to  elaborate  on  questions  1-20 
or  for  you  to  identify  any  other  concerns  that  you  might  have. 


APPENDIX  E 

SYSTEMS  QUESTIONNAIRE 


95 


Systems  Questionnaire 

The  following  questions  will  address  issues  concerning  the  physical  characteristics  and 
operation  of  tiie  various  systems  used  during  this  study.  Your  answers  to  these  questions 
should  be  made  from  the  standpoint  of  the  operational  utility  in  the  KC-13S  air  refueling 
mission.  Space  has  been  provided  for  comments  after  each  question.  If  fiuther  space  is 
needed,  please  continue  on  the  backside  of  the  page.  Do  not  feel  you  have  to  comment  on 
every  question.  When  asked  about  the  location  of  the  various  pieces  of  equipment,  please 
consider  the  location  of  the  equipment  in  relationship  to  how  you  answered  previous 
questions.  For  example,  it  would  be  impossible  to  put  the  CDU  and  a  warning  caution 
^visoiy  system  in  the  exact  same  location.  All  comments  and  ideas  you  may  have  to 
improve  the  operational  utility  of  the  equipment  and  reduce  crew  workload  will  be  greatly 
appreciated. 
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Analog  Flight  Instruments 

1.  Please  answer  Yes  (Y)  or  No  (N)  to  the  following  questions  for  each  of 
the  analog  flight  instruments  listed  below.  Please  comment  on  all  "No** 
answers. 


Size:  Is  the  size  of  the  instrument  adequate  for  the  application? 

Number:  Are  there  enough  of  the  instruments  (as  backups)  in  the  cockpit? 
Location:  Is  the  location  of  the  instrument  adequate? 

Necessity:  Is  the  the  instrument  necessary  or  critical? 


Instrument 


Number 


Location 


easily 


Attitude 

Director 

Indicalfii 


XADll 


Aitimetfii: 

Angle  of 
Attack  (AOA) 
Indicator 

Clock 

Indicated 

jUrspctd  (IAS) 

Mach 

Ifldicatnr 

Radio 

AitimtUr 


Radio 
Magnetic 
Indicator  (RMI) 


A 


A 


Vertical 
Velocity 
Indicator  .f 


2.  Should  the  backup  ADI  have  flight  direction  capability?  Yes  No 

Comments: _ 


3.  Wouid  a  digital  clock  adequately  replace  the  analog  clock?  Yes  No 


4.  Is  a  chronograph  (count  up)  function  needed?  Yes  No 

5.  Is  a  tinier  (count  down  )  function  needed?  Yes  No 

Comments  (3-5): _ 


6.  Should  the  Mach  and  Indicated  Airspeed  instruments  be  combined? 
Yes  No 


Comments: 


7.  Based  on  your  experience,  the  reliability  of  the  current  radio  altimeter  is 


a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Qxnpletely  acceptable 

Comments: _ 
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Digital  Warning  Caution  Advisory  (WCA)  Display 


1.  The  size  of  the  WCA  display  was _ ,  considering  its  possible 

application. 

Too  small  About  right  Too  large 

Comments: _ 


2.  The  size  of  the  characters  was _ ,  considering  viewing 

distance. 

Too  small  About  right  Too  large 

Comments: _ 


3.  The  dimming  (brightness)  capability  of  the  display  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderiine 

d.  Moderately  acceptable 

e.  Qxnpletely  acceptable 

Comments: _ 
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4.  The  location  of  the  display  was 


(Please  explain  below.) 


a.  (Completely  unacceptable 

b.  Motterately  unacceptable 

c.  Borderline 

(L  Moderately  acceptable 
e.  Oxnpletely  acc^table 

(Corrunents: _ 


5.  What  specific  warnings,  cautions,  and/or  advisories  would  you  like  to 
see  on  this  display? 

Comments: _ 


6.  What  warnings,  cautions,  and/or  advisories  would  you  prefer  not  be 
displayed? 

Comments:  _ _ _ 


7.  Was  the  Master  Caution  light  effective?  Yes  No 
Comments: _ 


8.  Would  you  prefer  a  muUiple>sensc»‘y  WCA  message  system? 


a.  No,  light  only. 

b.  Yes,  light  and  tone. 

c.  Yes,  li^t  and  voice. 

d.  Yes,  light ,  tone,  and  voice. 

Conunents: _ 


9.  Have  you  previously  been  exposed  to  digitally  displayed  engine 
instruments?  Yes  No 

Comments: _ 


10.  Do  you  feel  digitally  displayed  engine  instruments  are  necessary  for 
the  KC-135?  Yes  No 


Comments:. 


11.  Digital  Engine  Displays  have  the  capability  to  change  color,  alerting 

the  pilot  of  an  out*of*tolerance  condition.  This  capability _ 

my  answer  to  question  9? 

a.  Changes 

b.  Supports 

c.  Do^  not  affect 


Comments: 
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12.  Dieital  Engine  Displays  have  the  capability  to  integrate  WCA 

infomiation  with  engine  instruments.  This  capability _ my 

answer  to  question  9? 

a.  Changes 

b.  Supports 

c.  Do^  not  affect 

Comments: _ 


Control  Display  Unit  (CDU) 

This  section  asks  several  question  concerning  the  physical  characteristics  of  the 

CDU. 

1.  The  location  of  the  CDU  for  optimal  use  of  the  Mission  Management 
System  (MMS)  was _ ? 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Con^letely  acceptable 


Comments:. 


2.  The  dimming  (brightness)  capability  of  the  CDU  was 

a.  CtHnpletely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 

Comments: _ 


3.  The  size  of  the  alphanumeric  push  button  keys  is 
Too  small  About  right  Too  large 


Comments: 


4.  The  size  of  the  function  select  keys  (FSK)  is 


Too  small  About  right  Too  large 

Comments: _ 


5.  The  size  of  the  line  select  keys  (LSK)  is  _ 

Too  small  About  right  Too  large 

Comments: _ 


6.  The  tactile  feedback  for  switch  activation  without  gloves  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 
3.  Completely  acceptable 

Comments: _ 
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7.  The  tactile  feedback  for  switch  activation  with  gloves  on  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Bmderline 

d.  Motterately  acceptable 

e.  Completely  acceptable 

Comments: _ 


8.  The  readability  of  the  alphanumeric  key  legend/lettering  was 

a.  Completely  unacceptable 

b.  Moderately  unacc^table 

c.  Borderline 

d.  Moderately  accq>table 

e.  Con^letely  acc^table 

Comments: _ 


9.  The  readability  of  the  function  select  key  legend/lettering  was 


a.  Completely  unacceptable 

b.  Moderately  unaccept^le 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 

Corrunents: _ 
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10.  The  readability  of  the  CDU  lettering/symbols  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Con^letely  acc^table 

Comments: _ 


11.  Were  any  of  the  pages  cluttered?  Yes  No  (If  yes,  please  list  below.) 
Comments: _ 


12.  Did  you  experience  any  parallax/distortion  problems  with  any  of  the 
CDU  displays?  Yes  No  (If  yes,  please  explain  below.) 

Comments: _ ^ _ 
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Mission  Management  System  Operation 

This  section  will  address  the  operating  procedures  associated  with  the  functions  of 
the  Mission  Management  System  (MMS). 


1.  Were  the  symbol  conventions  (e.g.,  colons^  arrows,  etc.)  easy  to  use 
and  understand?  Yes  No  (If  no,  please  explain  below.) 

Comments: _ 


2.  The _  should  be  used  when  slewing  between  display  pages. 

a.  Up/Down  arrow  keys  b.  Right/Left  arrow  keys 
Comments: _ 


3.  The  flight  plan  loading  options  were _ .  (Please  explain  below.) 

a.  Ccanpletely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  C^on^letely  acceptable 


Comments:. 


4.  Flight  plan  modifications/correction  implementation  was 

a.  Oanpletely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d  Moderately  accq)table 
e.  Qni^letely  acceptable 

Comments: _ 


5.  Which  Direct-To  function  capability  did  you  prefer? 

a.  The  DIR  key  capability. 

b.  The  Waypoint  Summary  Page  capability. 

c.  Both  capabilities  were  equally  good 
d  Neither  ct^ability  was  adequate. 

e.  I  did  not  use  either  capability. 

(Please  explain  your  answer  in  furdier  detail  below.) 
Comments: _ 


6.  The  overfly  update  capability  of  this  system  was  _ 

a.  Oxnpletely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d  Moderately  acceptable 
e.  Ctxnpletely  acc^table 

(Please  explain  what  you  liked  or  disliked  about  this  capability  below.) 


7.  Was  the  Tacan  update  capability  of  this  system  easy  to  use?  Yes  No 


a.  Conqiletely  unaccqnable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Con^letely  acceptable 

(Please  explain  what  you  liked  or  dislilttd  about  diis  ctqubility  bdow.) 
Comments: _ 


8.  Was  the  holding  pattern  set-up  capability  beneficial?  Yes  No 


9.  The  ease  of  use  of  the  holding  pattern  capability  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  accqrtable 

e.  Completely  acceptable 

Comments  (8-9): _ 


10.  Was  the  Rendezvous  (Point  Parallel)  set>up  capability  beneficial? 

Yes  No 

11.  The  ease  of  implementation  of  the  Rendezvous  (Point  Parallel) 

capability  was _ . 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  CcMiq)letely  acceptable 

(Comments  (10-11): _ 


12.  Was  the  Intercept  set-up  capability  beneficial?  Yes  No 

13.  The  ease  of  implementation  of  the  Intercept  capability  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Coiiq)letely  acceptable 

Comments  (12-13): _ 
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14.  The  operational  utility  of  the  Mission  Computer  approach  used  in  this 
dnuilatcNr _ mission  difficulty/aircrew  workload?  Why? 


a.  Substantially  decreased 

b.  Moderately  decreased 

c.  Slightly  decreased 

d.  Did  not  increase  OT  decrease 

e.  Slightly  increased 

f.  Moderately  increased 

g.  Substantis^y  increased 

Comments : _ 


15.  Were  the  set-up  procedures  of  the  Mission  Computer  approach 

understandable  and  easy  to  use?  Yes  No  (If  no.  please  explain  below.) 

Comments: _ 


16.  Were  the  avionics  system  power  pages  understandable  and  easy  to  use? 
Yes  No  (If  no,  please  explain  below.) 

Comments : _ 
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Weather  Radar  System 

1.  The  location  of  the  radar  control  panel  was _ 

a.  Completely  unaccq>table 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  accq)table 

e.  Conqrletely  accqrtable 

Comments : _ 


2.  Should  each  pilot  be  given  a  separate  radar  range  control  to  allow  for 
the  monitoring  of  different  radar  ranges  at  the  same  time?  Yes  No 
Why? 

Comments : _ ^ _ 


3.  Were  the  various  range  selections  provided  adequate  for  mission 
accomplishment?  Yes  No  Why? 

Comments : _ ; _ 
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4.  The  radar  control  panel  numbering^ettering/nomenclature  was 

a.  Coo^letely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderiine 

d.  Modoately  acceptable 

e.  Qm^letely  accq)table 

Comments : _ 


5.  The  lighting  of  the  radar  control  panel  was _ for  inflict  use? 

a.  Completely  unaccqitable 

b.  Moderately  unacceptable 

c.  Borderiine 

d.  Moderately  accq>table 

e.  Conq)letely  aocq)tab]e 

Comments : _ 


6.  Are  there  any  functions  not  included  on  the  radar  control  panel  that  you 
feel  would  effectively  reduce  missitm  difficulty/aircrew  workload? 

Yes  No  Why? 

Comments: _ 


7.  Should  a  separate  dedicated  display  be  installed  for  the  radar  return 
presentation?  Yes  No  Why? 

Comments : _ 
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8.  Do  you  feel  the  need  for  a  weather  radar  as  necessary  for  mission 
accomplishment?  Yes  No  Would  a  monochrome  radar  presentation 
be  adequate?  Yes  No 

Comments : _ 


9.  Do  you  feel  the  need  still  exists  for  the  APN-69?  Yes  No 
Comments : _ 


10.  Do  you  prefer  the  Mission  Management  System  (MMS)  procedure  or 
the  current  APN*69  code  selectors  for  setting  the  beacon  code? 


a.  MMS  b.  APN-69 
Conunents : _ 


11.  Do  you  feel  the  Air-to-Air  tacan  capability  alone  would  be  sufficient  to 
perform  an  air  refueling  rendezvous?  Yes  No  Why? 


Comments 
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Electronic  Horizontal  Situation  Indicator  (EHSI) 


1.  The  location  of  the  EHSI  function  c<mtrol  panel  was _  for 

inflight  use? 

a.  Craipletely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Cbn^letely  acceptable 

Comments : _ 


2.  The  readability  of  the  function  control  panel  nomenclature  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acc^table 

Comments : _ 


3.  The  lighting  of  the  function  control  panel  was _ for  inflight  use? 

a.  Completely  unacceptable 

b.  Moderately  tinacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  COfiq>letely  acceptable 

Comments : _ 
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for 


4.  The  location  of  the  map  overlay  control  panel  was 
inflight  use? 

a.  Qxnpletely  unaccq)table 

b.  Moderately  unacceptable 

c.  Boideiline 

d.  Moderately  acceptable 

e.  Q}i]:q)letBly  acceptable 

Comments : _ 


5.  The  readability  of  the  map  overlay  control  panel  nomenclature  was 


a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 

Comments : _ 


6.  The  lighting  of  the  map  overlay  contnrf  pand  was _ for 

inflight  use. 

a.  Completely  unaccqnable 

b.  Moderately  unacc^table 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 


Comments 


7.  The  ability  to  overlay  waypoints,  navaids  (TacansA^ORs),  and  airports 

on  the  EHSI  map  display _  mission  difficulty/aircrew 

workload? 

a.  Substantially  decreased 

b.  Moderately  decreased 

c.  Slightly  decreased 

d.  Did  not  increase  or  decrease 

e.  Slightly  increased 

f.  Moderately  increased 

g.  Substanti^y  increased 

Comments : _ 


8.  Did  you  like  the  capability  to  display  either  a  full  compass  rose  or  an 
arc  segment  (60^)  individually  on  the  EHSI?  Yes  No  Why? 


Comments : 


9.  Do  you  feel  the  capability  to  display  the  selected  radio  aids  on  the  EHSI 
display  as  beneflcial?  Yes  No  Why? 


Comments : 


10.  The  EHSI  dimming  (brightness)  capability  was 


a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 


Comments 


Fuel  Savings  Advisory/Cockpit  Avionics  Systems 

(FS  A/CAS) 

Integrated  Fuel  Management  Panel  (IFMP) 

1.  The  IFMP  location  in  this  KC-135  crew  station  was  _ . 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 

Comments: _ 


2.  The  size  of  the  integrated  fuel  management  panel  was 
Too  small  About  right  Too  large 

Comments: _ 


3.  The  size  of  the  switches  was 


Too  small  About  right  Too  large 

Comments: _ 
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4.  The  spacing  between  switches  was  _ 

Too  small  About  right  Too  large 

Comments: _ 


5.  The  readability  of  the  low  pressure/low  fuel  annunciators  on  the  IFMP 
was 


a.  Ccnnpletely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acc^table 

Comments: _ 


6.  The  size  of  the  digital  fuel  quantity/CG  indicator  was 


Too  small  About  right  Too  large 


Comments: 


7.  The  color  of  the  digital  fuel  quantity/CG  readouts  was 


a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Coiiq)letely  acceptable 

Comments: _ 


8.  The  dimming  capability  of  the  digital  fuel  quantity/CG  readouts  was 


a.  Completely  unaccq)table 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acc^table 

Comments: _ 


9.  The  IFMP  lighting  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d  Moderately  acceptable 
e.  Completely  acceptable 

Comments: _ 
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10.  The  level  of  workload  required  to  monitor  fuel  flow  and  fhel  quantity 
information  was _ . 

a.  Extremely  difficult 

b.  Moderately  difficult 

c.  Borderiine 

d.  Moderately  easy 

e.  Extremely  easy 

Comments: _ 


11.  The  overall  operational  utility  of  the  IFMP  is 

a.  Completely  unaccq)table 

b.  Moderately  unacceptable 

c.  Borderiine 

d.  Moderately  accq)table 

e.  Completely  accq)table 

Comments: _ 


12.  What  suggested  changes,  if  any,  would  you  suggest  to  make  the  IFMP 
easier  to  operate? 


FSA/CAS  ICDU  Display  Formats 


1.  The  accessibility  of  the  FSA/CAS  ICDU  display,  in  context  with  the 
other  display  pages  in  the  menu  structure,  was _ 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Boideiiine 

d.  Moderately  acceptable 

e.  Coiiq>letely  acc^table 

Comments: _ 


2.  The  information  presented  on  this  display  was  _  _ .  What 

information  should  be  added/deleted?  (Please  explain  in  comments  section.) 

a.  (Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 

Comments: _ 


3.  The  FSA/CAS  ICDU  display  was  to  use.  What 

difficulties,  if  any,  were  encountered  with  this  unit? 

a.  Extremely  difficult 

b.  Moderately  difficult 

c.  Borderline 

d.  Moderately  easy 

e.  Extremely  easy 

Comments: _ 
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4.  The  workload  level  required  to  operate  the  FSA/CAS  using  the  ICDU 
for  fhel  management  tasks  was _ 

a.  Extremely  difficult 

b.  Moderately  difficult 

c.  Bordeiline 

d.  Moderately  easy 

e.  Extremely  easy 


Comments: 


Remote  Display  Unit  (RDU) 

1.  The  location  of  the  RDU  was _ 

a.  Completely  unacceptable 

b.  Modetately  unacceptable 

c.  Borderline 

d.  Moderately  accq)table 

e.  Ccxnpletely  aa;^table 

Comments: _ 


2.  The  size  of  the  RDU  command  airspeed  and  altitude  digital  readouts 
was  _ . 

Too  small  About  right  Too  large 


Comments: 


3.  The  color  of  the  RDU  command  airspeed  and  altitude  digital  readouts 
was  _ . 

a.  Qnnpletely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderiine 

d.  Moderately  acceptable 

e.  Completely  acc^table 


Comments: 


4.  The  dimming  capability  of  the  RDU  command  airspeed  and  altitude 
digital  readouts  was _ 

a.  Coaq>letely  uiiaccq)table 

b.  Modoately  unacc^table 

c.  Boideiiine 

d.  Moderately  acceptable 

e.  Completely  acceptable 

Comments: _ 


5.  What  information  should  be  added/deleted  from  this  RDU? 


Comments: 


Remote  Radio  Unit  (RRU) 


Communication  Readouts 

1.  The  operational  utility  of  the  RRU  was 

a.  Qnnpletely  unaccq)table 

b.  Moderately  unacceptable 

c.  Borderiine 

d.  Moderately  acceptable 

e.  Completely  acceptable 

Comments: _ 


2.  The  location  of  the  RRU  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d  Moderately  acceptable 
e.  Completely  acceptable 

Comments: _ 


3.  The  size  of  the  RRU  digital  readouts  was _ 

Too  small  About  ri^t  Too  large 

Comments: _ 
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4.  The  color  of  the  RRU  digital  readouts  was 


a.  Conq>tetely  unaccqKable 

b.  Moderately  unaccq)table 

c.  Borderline 

d.  Moderately  aocqitable 

e.  Completely  aocq)table 

Comments: _ 


5.  The  brightness  of  the  RRU  digital  readouts  was 

a.  CompletBly  unaccq)table 

b.  Moderately  unaccqitable 

c.  Borderline 

d.  Moderately  accq)table 

e.  Con^letely  accq)table 

Comments: _ 


Ground  Mission  Planning  System  (MPS) 

1.  The  existing  ground  MPS  was _ 

a.  Completely  unacceptable 

b.  Mot^ttly  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 

Comments; _ 


2.  To  what  extent  is  the  aircrew  involved  in  the  initial  mission  planning  on 
the  MPS  using  the  Z-248? 

Comments: _ 


3.  What  changes,  if  any,  would  you  make  to  this  system. 
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Electronic  Attitude  Director  Indicator  (EADI) 


1.  The  EADI  dimming  (brightness)  capability  was 

a.  Completely  unacceptable 

b.  Modaately  unacceptable 

c.  Boideiline 

d.  Moderately  accq)table 

e.  Coixq>letely  acceptable 

Coixunents: _ 


2.  The  location  of  the  EADI  Decision  Height  (DH)  set  knob  was 
_ for  inflight  iSe. 

a.  Gnnpletely  unaccepuible 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  accqxtable 

e.  Con^letely  accqxtable 

Comments:  _ _ _ _ 


3.  The  letters  "DH**  appearing  in  the  left  center  of  the  EADI  when  radar 

altitude  went  below  the  decision  height  setting  was  _ .  (Please 

explain  below.) 

a.  Helpful  aiKi  sufficient 

b.  Helpful,  but  not  sufficient. 

c.  Not  helpful 

Comments: _ 
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4.  The  digital  presentation  for  the  Decision  Height  setting  was 


«  a.  Gxnpletely  unacceptable 

b.  Moderately  unaccepiiable 

c.  Bordeiiine 

*  d.  Moderately  acceptable 

e.  Ccm^letely  acc^table 

Comments: _ 


5.  The  EADI  has  digital  information  displayed  for  Indicated  airspeed  (IAS 
An^le  of  Attack,  Barometric  Pressure,  Vertical  Velocity  (VV),  and 
radio  altimeter  altitude.  Please  answer  Yes  (Y)  or  No  (N)  to  the 
following  questions 

Size:  Is  the  size  of  the  information  adequate  for  its  application? 

Location:  Is  the  location  of  the  information  adequate? 

Necessity:  Is  the  displayed  information  necessary  or  critical? 


Interiaatiqn _ Siu _ Location _ Necessity 

Angle  of 

Attack  (AQA)  _  _  _ 

Barometric 

Pressure 

Altitude  IBI  _  _  _ 

Indicated 

Airspeed  (1)  _  _ _  _ 

Radio 

Altimdfi  (RA)  _  _  _ 

Vertical 

Velocity  fVVI  _  _  _ 


Comments: 
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6.  Should  a  speed  deviation  indicator  scale  be  placed  onto  the  EADI? 


a.  Yes,  it  would  be  very  helpful. 

b.  Yes,  it  would  be  moderately  helpful. 

c.  Yes,  it  would  be  somewhat  helpful. 

d.  No,  it  would  not  be  at  all  helpful. 

Comments: _ 


7.  The  size  of  the  EADI  was 

a.  Completely  unacceptable 

b.  Moderately  unacceptable 

c.  Borderline 

d.  Moderately  acceptable 

e.  Completely  acceptable 


Comments: 


8.  The  colors  used  for  the  EADI  were 

a.  Completely  unacceptable 

b.  Moderately  unacc^table 

c.  Borderline 

d.  Moderately  acceptable 

e.  Ccanpletely  acceptable 


Cbmments: 


9.  Identify  any  specific  changes  you  might  have  concerning  the  EADI: 
Comments: _ 


